[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:00 dist-test-slave-dn29 krb5kdc[5413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:00 dist-test-slave-dn29 krb5kdc[5413](info): set up 2 sockets
Sep 23 20:46:00 dist-test-slave-dn29 krb5kdc[5413](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 20:46:01.469161 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.006s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:01 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:01.682677 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43993
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:43993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:02.230647 5426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:02.232236 5426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:02.232336 5426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:02.240664 5426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:02.240783 5426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:02.240829 5426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:02.240880 5426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:02.240922 5426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:02.251124 5426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:43993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:02.255026 5426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:02.257714 5426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:02.284511 5431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:02.284523 5432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:02.284426 5434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:02.284754 5426 server_base.cc:1034] running on GCE node
I20260923 20:46:02.288275 5426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:02.289994 5426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:02.291322 5426 hybrid_clock.cc:648] HybridClock initialized: now 1790196362291252 us; error 71 us; skew 500 ppm
I20260923 20:46:02.292548 5426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:02 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196362, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:02.296319 5426 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:02.298074 5426 webserver.cc:460] Webserver started at http://127.5.71.190:39475/ using document root <none> and password file <none>
I20260923 20:46:02.299971 5426 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:02.300133 5426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:02.300486 5426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:02.311041 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "3826c9ac64b74e3eac5927ba9927ab8a"
format_stamp: "Formatted at 2026-09-23 20:46:02 on dist-test-slave-dn29"
server_key: "1c3c5f31aba119c7ff9cd83f631698c3"
server_key_iv: "58f35b1a08cda8d13612d398a934ae8a"
server_key_version: "encryptionkey@0"
I20260923 20:46:02.313959 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "3826c9ac64b74e3eac5927ba9927ab8a"
format_stamp: "Formatted at 2026-09-23 20:46:02 on dist-test-slave-dn29"
server_key: "1c3c5f31aba119c7ff9cd83f631698c3"
server_key_iv: "58f35b1a08cda8d13612d398a934ae8a"
server_key_version: "encryptionkey@0"
I20260923 20:46:02.324146 5426 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.006s
I20260923 20:46:02.336285 5441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:02.338030 5426 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260923 20:46:02.338335 5426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "3826c9ac64b74e3eac5927ba9927ab8a"
format_stamp: "Formatted at 2026-09-23 20:46:02 on dist-test-slave-dn29"
server_key: "1c3c5f31aba119c7ff9cd83f631698c3"
server_key_iv: "58f35b1a08cda8d13612d398a934ae8a"
server_key_version: "encryptionkey@0"
I20260923 20:46:02.338651 5426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:02.366690 5426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:02.371809 5426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:02.372112 5426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:02.395526 5426 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:43993
I20260923 20:46:02.395578 5492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:43993 every 8 connection(s)
I20260923 20:46:02.398736 5426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:02.408047 5493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:02.408390 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 5426
I20260923 20:46:02.408716 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:02.410789 5406 external_mini_cluster.cc:1428] Setting key 3616751b818b33edd5b6f215493cb2e9
I20260923 20:46:02.428279 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a: Bootstrap starting.
I20260923 20:46:02.435865 5493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:02.440112 5493 log.cc:824] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:02.444473 5493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a: No bootstrap required, opened a new log
Sep 23 20:46:02 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:02.462894 5493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43993 } }
I20260923 20:46:02.463590 5493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:02.463887 5493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3826c9ac64b74e3eac5927ba9927ab8a, State: Initialized, Role: FOLLOWER
I20260923 20:46:02.464918 5493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43993 } }
I20260923 20:46:02.465391 5493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:02.465687 5493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:02.466871 5493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:02.468703 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:02.416868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.5.71.190:43993)
0923 20:46:02.417720 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:02.418497 (+ 777us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:02.419231 (+ 734us) server_negotiation.cc:407] Connection header received
0923 20:46:02.428575 (+ 9344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:02.428607 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:02.429311 (+ 704us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:02.429810 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:02.433440 (+ 3630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:02.439681 (+ 6241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:02.444781 (+ 5100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:02.445769 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:02.446122 (+ 353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:02.455460 (+ 9338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:02.455471 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:02.455492 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:02.455565 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:02.461106 (+ 5541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:02.462474 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:02.462483 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:02.462499 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:02.462857 (+ 358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:02.463841 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:02.463847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:02.463850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:02.464765 (+ 915us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:02.465285 (+ 520us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:02.466187 (+ 902us) server_negotiation.cc:299] Negotiation successful
0923 20:46:02.466566 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":391,"threads_started":1}
I20260923 20:46:02.470969 5493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43993 } }
I20260923 20:46:02.472339 5493 leader_election.cc:302] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3826c9ac64b74e3eac5927ba9927ab8a; no voters:
I20260923 20:46:02.473129 5493 leader_election.cc:288] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:02.473332 5498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:02.474107 5498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [term 1 LEADER]: Becoming Leader. State: Replica: 3826c9ac64b74e3eac5927ba9927ab8a, State: Running, Role: LEADER
I20260923 20:46:02.474854 5498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43993 } }
I20260923 20:46:02.478883 5493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:02.498500 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43993 } } }
I20260923 20:46:02.499064 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:02.498847 5500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3826c9ac64b74e3eac5927ba9927ab8a. Latest consensus state: current_term: 1 leader_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3826c9ac64b74e3eac5927ba9927ab8a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43993 } } }
I20260923 20:46:02.499204 5500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a [sys.catalog]: This master's current role is: LEADER
W20260923 20:46:02.500869 5513 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:46:02.500978 5513 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:46:02.501113 5515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:02.502849 5515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:02.511300 5515 catalog_manager.cc:1347] Generated new cluster ID: a14a585cacee43e5b1a46d9f8cb274d5
I20260923 20:46:02.512719 5515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:02.567672 5515 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:02.569533 5515 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:02.580268 5515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3826c9ac64b74e3eac5927ba9927ab8a: Generated new TSK 0
I20260923 20:46:02.581314 5515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:02.636912 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43993
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:03.063637 5519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:03.064139 5519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:03.064419 5519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:03.074928 5519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:03.075165 5519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:03.075440 5519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:03.089502 5519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:03.090931 5519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:03.092729 5519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:03.113083 5524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:03.113391 5525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:03.113817 5527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:03.115111 5519 server_base.cc:1034] running on GCE node
I20260923 20:46:03.115991 5519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:03.117373 5519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:03.118746 5519 hybrid_clock.cc:648] HybridClock initialized: now 1790196363118560 us; error 68 us; skew 500 ppm
I20260923 20:46:03.119261 5519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:03 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196363, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:03.124270 5519 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:03.126503 5519 webserver.cc:460] Webserver started at http://127.5.71.129:43799/ using document root <none> and password file <none>
I20260923 20:46:03.127604 5519 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:03.127831 5519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:03.128400 5519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:03.132545 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "1cd36ea8caf84845a537c9690882cf62"
format_stamp: "Formatted at 2026-09-23 20:46:03 on dist-test-slave-dn29"
server_key: "5df5582fc4063fc931aa5d1e47142744"
server_key_iv: "0f10f7e5860dd2a92b342393544e7474"
server_key_version: "encryptionkey@0"
I20260923 20:46:03.133582 5519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "1cd36ea8caf84845a537c9690882cf62"
format_stamp: "Formatted at 2026-09-23 20:46:03 on dist-test-slave-dn29"
server_key: "5df5582fc4063fc931aa5d1e47142744"
server_key_iv: "0f10f7e5860dd2a92b342393544e7474"
server_key_version: "encryptionkey@0"
I20260923 20:46:03.141325 5519 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260923 20:46:03.146497 5534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:03.148478 5519 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 20:46:03.148730 5519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "1cd36ea8caf84845a537c9690882cf62"
format_stamp: "Formatted at 2026-09-23 20:46:03 on dist-test-slave-dn29"
server_key: "5df5582fc4063fc931aa5d1e47142744"
server_key_iv: "0f10f7e5860dd2a92b342393544e7474"
server_key_version: "encryptionkey@0"
I20260923 20:46:03.149034 5519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:03.168594 5519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:03.174979 5519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:03.175447 5519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:03.177142 5519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:03.179229 5519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:03.179384 5519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:03.179643 5519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:03.179778 5519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:03.256610 5519 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:39061
I20260923 20:46:03.256760 5646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:39061 every 8 connection(s)
I20260923 20:46:03.258102 5519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:03.261744 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 5519
I20260923 20:46:03.261984 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:03.262727 5406 external_mini_cluster.cc:1428] Setting key 77df7205ee2c15e31b8077346d3e0d6e
Sep 23 20:46:03 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196363, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:03.286967 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:03.262999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:34605 (local address 127.5.71.190:43993)
0923 20:46:03.263595 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:03.263602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:03.265288 (+ 1686us) server_negotiation.cc:407] Connection header received
0923 20:46:03.267376 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:03.267382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:03.267484 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:03.267612 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:03.269499 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:03.270338 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:03.273451 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:03.273965 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:03.274079 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:03.279425 (+ 5346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:03.279438 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:03.279442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:03.279479 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:03.282406 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:03.283502 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:03.283506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:03.283509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:03.283650 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:03.284357 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:03.284361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:03.284364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:03.284675 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:03.284798 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:03.286096 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0923 20:46:03.286404 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":358,"threads_started":1}
I20260923 20:46:03.288043 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:03.263968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43993 (local address 127.5.71.129:34605)
0923 20:46:03.265040 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:03.265136 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:03.267047 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:03.267950 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:03.267959 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:03.268609 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:03.269284 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:03.269304 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:03.270534 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:03.270544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:03.273207 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:03.273230 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:03.274245 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:03.274256 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:03.274650 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:03.275516 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:03.275557 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:03.279070 (+ 3513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:03.282572 (+ 3502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:03.282581 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:03.282598 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:03.283333 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:03.283923 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:03.283927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:03.283930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:03.284194 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:03.284864 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:03.284896 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:03.285430 (+ 534us) client_negotiation.cc:769] Sending connection context
0923 20:46:03.285926 (+ 496us) client_negotiation.cc:240] Negotiation successful
0923 20:46:03.286466 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":363,"threads_started":1}
I20260923 20:46:03.290094 5647 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43993
I20260923 20:46:03.290520 5647 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:03.291628 5647 heartbeater.cc:511] Master 127.5.71.190:43993 requested a full tablet report, sending...
I20260923 20:46:03.295902 5458 ts_manager.cc:194] Registered new tserver with Master: 1cd36ea8caf84845a537c9690882cf62 (127.5.71.129:39061)
I20260923 20:46:03.298642 5458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:34605
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:03.313262 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43993
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:03.870391 5653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:03.870673 5653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:03.870779 5653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:03.878118 5653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:03.878234 5653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:03.878350 5653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:03.888500 5653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:03.889895 5653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:03.891637 5653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:03.906999 5658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:03.907164 5659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:03.907164 5661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:03.908154 5653 server_base.cc:1034] running on GCE node
I20260923 20:46:03.908880 5653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:03.909883 5653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:03.911114 5653 hybrid_clock.cc:648] HybridClock initialized: now 1790196363911066 us; error 38 us; skew 500 ppm
I20260923 20:46:03.911502 5653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:03 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196363, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:03.915045 5653 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:03.916729 5653 webserver.cc:460] Webserver started at http://127.5.71.130:40973/ using document root <none> and password file <none>
I20260923 20:46:03.917447 5653 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:03.917577 5653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:03.917904 5653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:03.920979 5653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "912ff215e4bd4d0481b05dbfc615ed37"
format_stamp: "Formatted at 2026-09-23 20:46:03 on dist-test-slave-dn29"
server_key: "76755617097021d25fb8cccc83660da7"
server_key_iv: "baffda16982490c01e1e30ee246db614"
server_key_version: "encryptionkey@0"
I20260923 20:46:03.921815 5653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "912ff215e4bd4d0481b05dbfc615ed37"
format_stamp: "Formatted at 2026-09-23 20:46:03 on dist-test-slave-dn29"
server_key: "76755617097021d25fb8cccc83660da7"
server_key_iv: "baffda16982490c01e1e30ee246db614"
server_key_version: "encryptionkey@0"
I20260923 20:46:03.927369 5653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:03.931185 5668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:03.932726 5653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:03.932866 5653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "912ff215e4bd4d0481b05dbfc615ed37"
format_stamp: "Formatted at 2026-09-23 20:46:03 on dist-test-slave-dn29"
server_key: "76755617097021d25fb8cccc83660da7"
server_key_iv: "baffda16982490c01e1e30ee246db614"
server_key_version: "encryptionkey@0"
I20260923 20:46:03.933038 5653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:03.965294 5653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:03.971422 5653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:03.971984 5653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:03.973683 5653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:03.975721 5653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:03.975905 5653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260923 20:46:03.976106 5653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:03.976248 5653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:04.033469 5653 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:41587
I20260923 20:46:04.033553 5780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:41587 every 8 connection(s)
I20260923 20:46:04.034842 5653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:04.036536 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 5653
I20260923 20:46:04.036828 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:04.037467 5406 external_mini_cluster.cc:1428] Setting key 5c5f7c3d235a0bf87592e6e6a94c278d
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196363, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:04.060626 5783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.037963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:43593 (local address 127.5.71.190:43993)
0923 20:46:04.039408 (+ 1445us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:04.039412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:04.040169 (+ 757us) server_negotiation.cc:407] Connection header received
0923 20:46:04.042058 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:04.042064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:04.042159 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:04.042282 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:04.043925 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.044698 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.047518 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.048126 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.048237 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.052949 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:04.052956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:04.052960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:04.052998 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:04.056394 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.057393 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.057398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.057401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.057532 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.058125 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.058131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.058137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.058612 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:04.058785 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:04.059647 (+ 862us) server_negotiation.cc:299] Negotiation successful
0923 20:46:04.059974 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1338,"thread_start_us":1202,"threads_started":1}
I20260923 20:46:04.061347 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.038665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43993 (local address 127.5.71.130:43593)
0923 20:46:04.040019 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:04.040066 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:04.041773 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:04.042548 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:04.042557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:04.043101 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:04.043708 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:04.043728 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.044893 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.044901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:04.047296 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:04.047315 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.048389 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.048399 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:04.048745 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.049528 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:04.049565 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:04.052663 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:04.056575 (+ 3912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:04.056586 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:04.056609 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:04.057215 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:04.057680 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:04.057689 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:04.057696 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:04.057981 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:04.058784 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:04.058791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:04.059210 (+ 419us) client_negotiation.cc:769] Sending connection context
0923 20:46:04.059488 (+ 278us) client_negotiation.cc:240] Negotiation successful
0923 20:46:04.060036 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":798,"threads_started":1}
I20260923 20:46:04.063290 5781 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43993
I20260923 20:46:04.063648 5781 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:04.064581 5781 heartbeater.cc:511] Master 127.5.71.190:43993 requested a full tablet report, sending...
I20260923 20:46:04.066259 5458 ts_manager.cc:194] Registered new tserver with Master: 912ff215e4bd4d0481b05dbfc615ed37 (127.5.71.130:41587)
I20260923 20:46:04.067183 5458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:43593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:04.084928 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43993
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:46:04.302771 5647 heartbeater.cc:503] Master 127.5.71.190:43993 was elected leader, sending a full tablet report...
W20260923 20:46:04.512626 5787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:04.513298 5787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:04.513826 5787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:04.527740 5787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:04.527954 5787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:04.528190 5787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:04.538919 5787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43993
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:04.540401 5787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:04.541760 5787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:04.559149 5793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:04.559213 5792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:04.559190 5795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:04.559989 5787 server_base.cc:1034] running on GCE node
I20260923 20:46:04.560665 5787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:04.561682 5787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:04.562942 5787 hybrid_clock.cc:648] HybridClock initialized: now 1790196364562868 us; error 61 us; skew 500 ppm
I20260923 20:46:04.563426 5787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:04.566730 5787 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:04.568495 5787 webserver.cc:460] Webserver started at http://127.5.71.131:37111/ using document root <none> and password file <none>
I20260923 20:46:04.569303 5787 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:04.569406 5787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:04.569664 5787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:04.572715 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "9f8fcde0db1c420fb293039b0d9c264c"
format_stamp: "Formatted at 2026-09-23 20:46:04 on dist-test-slave-dn29"
server_key: "aa0d0dcf5815cb66e3dfa27ccaa10e3e"
server_key_iv: "006e18dc1bceb1d7b53e9bea074cf5f4"
server_key_version: "encryptionkey@0"
I20260923 20:46:04.573457 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "9f8fcde0db1c420fb293039b0d9c264c"
format_stamp: "Formatted at 2026-09-23 20:46:04 on dist-test-slave-dn29"
server_key: "aa0d0dcf5815cb66e3dfa27ccaa10e3e"
server_key_iv: "006e18dc1bceb1d7b53e9bea074cf5f4"
server_key_version: "encryptionkey@0"
I20260923 20:46:04.579010 5787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:04.582722 5802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:04.584426 5787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:04.584584 5787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "9f8fcde0db1c420fb293039b0d9c264c"
format_stamp: "Formatted at 2026-09-23 20:46:04 on dist-test-slave-dn29"
server_key: "aa0d0dcf5815cb66e3dfa27ccaa10e3e"
server_key_iv: "006e18dc1bceb1d7b53e9bea074cf5f4"
server_key_version: "encryptionkey@0"
I20260923 20:46:04.584758 5787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:04.610903 5787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:04.615912 5787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:04.616187 5787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:04.617237 5787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:04.618623 5787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:04.618701 5787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:04.618791 5787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:04.618848 5787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:04.656075 5787 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:43529
I20260923 20:46:04.656266 5914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:43529 every 8 connection(s)
I20260923 20:46:04.657498 5787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:04.665997 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 5787
I20260923 20:46:04.666196 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:04.666641 5406 external_mini_cluster.cc:1428] Setting key 802727e5723fe14cc9f58856e08b2414
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:04.680907 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.660567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:47545 (local address 127.5.71.190:43993)
0923 20:46:04.661088 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:04.661093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:04.662086 (+ 993us) server_negotiation.cc:407] Connection header received
0923 20:46:04.664166 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:04.664173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:04.664278 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:04.664398 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:04.666187 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.667288 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.669746 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.670455 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.670588 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.674494 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:04.674500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:04.674503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:04.674537 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:04.677217 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.678091 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.678096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.678099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.678246 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.678825 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.678829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.678832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.679227 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:04.679372 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:04.680389 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0923 20:46:04.680557 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":282,"threads_started":1}
I20260923 20:46:04.682010 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.661093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43993 (local address 127.5.71.131:47545)
0923 20:46:04.661891 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:04.661931 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:04.663890 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:04.664643 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:04.664652 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:04.665158 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:04.665871 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:04.665896 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.667466 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.667470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:04.669577 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:04.669593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.670767 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.670775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:04.670981 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.671842 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:04.671871 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:04.674261 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:04.677366 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:04.677375 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:04.677389 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:04.677924 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:04.678435 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:04.678438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:04.678441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:04.678633 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:04.679359 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:04.679400 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:04.679917 (+ 517us) client_negotiation.cc:769] Sending connection context
0923 20:46:04.680225 (+ 308us) client_negotiation.cc:240] Negotiation successful
0923 20:46:04.680747 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":326,"threads_started":1}
I20260923 20:46:04.683490 5915 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43993
I20260923 20:46:04.683889 5915 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:04.684624 5915 heartbeater.cc:511] Master 127.5.71.190:43993 requested a full tablet report, sending...
I20260923 20:46:04.686066 5458 ts_manager.cc:194] Registered new tserver with Master: 9f8fcde0db1c420fb293039b0d9c264c (127.5.71.131:43529)
I20260923 20:46:04.687139 5458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:47545
I20260923 20:46:04.696203 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:04.746742 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.728489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.5.71.129:39061)
0923 20:46:04.729339 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:04.729349 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:04.729384 (+ 35us) server_negotiation.cc:407] Connection header received
0923 20:46:04.729611 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:04.729620 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:04.729872 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:04.730058 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:04.731719 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.732904 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.734466 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.735112 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.735274 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.740034 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:04.740047 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:04.740065 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:04.740110 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:04.743402 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.744305 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.744314 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.744317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.744453 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.744883 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.744890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.744894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.745284 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:04.745622 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:04.746018 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 20:46:04.746316 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":433,"threads_started":1}
W20260923 20:46:04.748468 5562 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39974
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:04.805061 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.785322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.5.71.129:39061)
0923 20:46:04.785596 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:04.785607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:04.785978 (+ 371us) server_negotiation.cc:407] Connection header received
0923 20:46:04.786223 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:04.786228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:04.786329 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:04.786455 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:04.788192 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.789254 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.790700 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.791367 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.791520 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.797590 (+ 6070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:04.797597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:04.797603 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:04.797650 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:04.801493 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.802405 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.802419 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.802422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.802650 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.803085 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.803091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.803095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.803416 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:04.803766 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:04.804230 (+ 464us) server_negotiation.cc:299] Negotiation successful
0923 20:46:04.804544 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260923 20:46:04.806906 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 5519
I20260923 20:46:04.824358 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.808076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.5.71.129:39061)
0923 20:46:04.808400 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:04.808406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:04.808641 (+ 235us) server_negotiation.cc:407] Connection header received
0923 20:46:04.808861 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:04.808870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:04.808937 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:04.809048 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:04.811286 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.812461 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.814521 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.815362 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.815518 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.818188 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:04.818194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:04.818197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:04.818231 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:04.821435 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.822213 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.822220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.822222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.822367 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.822777 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.822783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.822786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.823063 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:04.823352 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:04.823660 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 20:46:04.823956 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 20:46:04.825325 5642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:04.916697 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 5519
I20260923 20:46:04.933341 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 5653
Sep 23 20:46:04 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:04.951304 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:04.934435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.5.71.130:41587)
0923 20:46:04.935068 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:04.935076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:04.935102 (+ 26us) server_negotiation.cc:407] Connection header received
0923 20:46:04.935164 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:04.935171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:04.935395 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:04.935526 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:04.936886 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.937944 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.939379 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:04.940078 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:04.940183 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:04.944873 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:04.944886 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:04.944904 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:04.944952 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:04.948578 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.949268 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.949276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.949279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.949406 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:04.949776 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:04.949780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:04.949782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:04.950141 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:04.950412 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:04.950705 (+ 293us) server_negotiation.cc:299] Negotiation successful
0923 20:46:04.950936 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":393,"threads_started":1}
I20260923 20:46:04.952939 5776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:05.044035 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 5653
I20260923 20:46:05.065788 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 5787
Sep 23 20:46:05 dist-test-slave-dn29 krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:05.083909 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:05.066912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.5.71.131:43529)
0923 20:46:05.067509 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:05.067516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:05.067547 (+ 31us) server_negotiation.cc:407] Connection header received
0923 20:46:05.067703 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:05.067713 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:05.068047 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:05.068181 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:05.069428 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:05.070593 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:05.072126 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:05.072734 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:05.072870 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:05.077730 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:05.077745 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:05.077763 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:05.077867 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:05.081075 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:05.081894 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:05.081901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:05.081903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:05.082039 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:05.082396 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:05.082399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:05.082401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:05.082743 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:05.083058 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:05.083298 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 20:46:05.083513 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":254,"threads_started":1}
I20260923 20:46:05.085391 5910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:05.171483 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 5787
I20260923 20:46:05.187778 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 5426
I20260923 20:46:05.189056 5488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:05.293516 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 5426
2026-09-23T20:46:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5065 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:05 dist-test-slave-dn29 krb5kdc[5937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:05 dist-test-slave-dn29 krb5kdc[5937](info): set up 2 sockets
Sep 23 20:46:05 dist-test-slave-dn29 krb5kdc[5937](info): commencing operation
krb5kdc: starting...
W20260923 20:46:06.488415 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:06 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:06.685674 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40831
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:07.018666 5950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:07.019055 5950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:07.019203 5950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:07.027863 5950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:07.028002 5950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:07.028096 5950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:07.028180 5950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:07.028241 5950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:07.038736 5950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:07.040304 5950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:07.041786 5950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:07.054986 5955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:07.055022 5956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:07.055022 5958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:07.056216 5950 server_base.cc:1034] running on GCE node
I20260923 20:46:07.057111 5950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:07.058590 5950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:07.059916 5950 hybrid_clock.cc:648] HybridClock initialized: now 1790196367059864 us; error 43 us; skew 500 ppm
I20260923 20:46:07.060298 5950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:07 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196367, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:07.063628 5950 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:07.065457 5950 webserver.cc:460] Webserver started at http://127.5.71.190:41951/ using document root <none> and password file <none>
I20260923 20:46:07.066236 5950 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:07.066387 5950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:07.066705 5950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:07.069993 5950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "59ebdb93a07743de908efb2a705e4cc6"
format_stamp: "Formatted at 2026-09-23 20:46:07 on dist-test-slave-dn29"
server_key: "851a6ca3b598b023ffd8d4d8759823e8"
server_key_iv: "039dddeb215cc62d5213f5cf6db3aabc"
server_key_version: "encryptionkey@0"
I20260923 20:46:07.070806 5950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "59ebdb93a07743de908efb2a705e4cc6"
format_stamp: "Formatted at 2026-09-23 20:46:07 on dist-test-slave-dn29"
server_key: "851a6ca3b598b023ffd8d4d8759823e8"
server_key_iv: "039dddeb215cc62d5213f5cf6db3aabc"
server_key_version: "encryptionkey@0"
I20260923 20:46:07.076783 5950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:46:07.080924 5965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:07.082690 5950 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260923 20:46:07.082926 5950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "59ebdb93a07743de908efb2a705e4cc6"
format_stamp: "Formatted at 2026-09-23 20:46:07 on dist-test-slave-dn29"
server_key: "851a6ca3b598b023ffd8d4d8759823e8"
server_key_iv: "039dddeb215cc62d5213f5cf6db3aabc"
server_key_version: "encryptionkey@0"
I20260923 20:46:07.083160 5950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:07.101332 5950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:07.106599 5950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:07.106974 5950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:07.128153 5950 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:40831
I20260923 20:46:07.128207 6016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:40831 every 8 connection(s)
I20260923 20:46:07.129730 5950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:07.134168 6017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:07.135013 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 5950
I20260923 20:46:07.135278 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:07.136215 5406 external_mini_cluster.cc:1428] Setting key af3046899fb29a09d5f2fef25fb209c2
I20260923 20:46:07.150002 6017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6: Bootstrap starting.
Sep 23 20:46:07 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:07.154053 6017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:07.155531 6017 log.cc:824] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:07.159261 6017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6: No bootstrap required, opened a new log
I20260923 20:46:07.162760 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:07.138485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.5.71.190:40831)
0923 20:46:07.139324 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:07.139337 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:07.139381 (+ 44us) server_negotiation.cc:407] Connection header received
0923 20:46:07.141841 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:07.141872 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:07.142338 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:07.142709 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:07.144574 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:07.145981 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:07.147904 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:07.148638 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:07.148778 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:07.153704 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:07.153719 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:07.153744 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:07.153796 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:07.157921 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:07.158659 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:07.158668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:07.158676 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:07.158888 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:07.159240 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:07.159250 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:07.159254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:07.160055 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:07.160356 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:07.160817 (+ 461us) server_negotiation.cc:299] Negotiation successful
0923 20:46:07.161315 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":372,"threads_started":1}
I20260923 20:46:07.168762 6017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ebdb93a07743de908efb2a705e4cc6" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40831 } }
I20260923 20:46:07.169127 6017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:07.169209 6017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ebdb93a07743de908efb2a705e4cc6, State: Initialized, Role: FOLLOWER
I20260923 20:46:07.169890 6017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ebdb93a07743de908efb2a705e4cc6" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40831 } }
I20260923 20:46:07.170251 6017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:07.170423 6017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:07.170614 6017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:07.172989 6017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ebdb93a07743de908efb2a705e4cc6" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40831 } }
I20260923 20:46:07.173606 6017 leader_election.cc:302] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59ebdb93a07743de908efb2a705e4cc6; no voters:
I20260923 20:46:07.174319 6017 leader_election.cc:288] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:07.174480 6022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:07.175174 6022 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [term 1 LEADER]: Becoming Leader. State: Replica: 59ebdb93a07743de908efb2a705e4cc6, State: Running, Role: LEADER
I20260923 20:46:07.175853 6022 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ebdb93a07743de908efb2a705e4cc6" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40831 } }
I20260923 20:46:07.176481 6017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:07.178820 6024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59ebdb93a07743de908efb2a705e4cc6. Latest consensus state: current_term: 1 leader_uuid: "59ebdb93a07743de908efb2a705e4cc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ebdb93a07743de908efb2a705e4cc6" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40831 } } }
I20260923 20:46:07.178820 6023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59ebdb93a07743de908efb2a705e4cc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ebdb93a07743de908efb2a705e4cc6" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40831 } } }
I20260923 20:46:07.179296 6024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:07.179296 6023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:07.180223 6030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:07.190976 6030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:07.200488 6030 catalog_manager.cc:1347] Generated new cluster ID: 7b771f29f64b4f87b1974df94528ac48
I20260923 20:46:07.200634 6030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:07.216148 6030 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:07.218142 6030 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:07.234977 6030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59ebdb93a07743de908efb2a705e4cc6: Generated new TSK 0
I20260923 20:46:07.236193 6030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:07.289316 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40831
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:07.624213 6043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:07.624517 6043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:07.624675 6043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:07.632161 6043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:07.632287 6043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:07.632486 6043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:07.642578 6043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:07.644011 6043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:07.645387 6043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:07.659267 6048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:07.659322 6049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:07.659263 6051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:07.660158 6043 server_base.cc:1034] running on GCE node
I20260923 20:46:07.660820 6043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:07.661813 6043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:07.663049 6043 hybrid_clock.cc:648] HybridClock initialized: now 1790196367663006 us; error 43 us; skew 500 ppm
I20260923 20:46:07.663388 6043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:07 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196367, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:07.666797 6043 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:07.668466 6043 webserver.cc:460] Webserver started at http://127.5.71.129:33979/ using document root <none> and password file <none>
I20260923 20:46:07.669253 6043 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:07.669412 6043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:07.669739 6043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:07.673085 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "8c9d6c30b988406f9347f51c8dcc0159"
format_stamp: "Formatted at 2026-09-23 20:46:07 on dist-test-slave-dn29"
server_key: "44de56ccf17be6ee525b2762714c8156"
server_key_iv: "62080f9f7fb86e549fb2a7a3f15cc6d7"
server_key_version: "encryptionkey@0"
I20260923 20:46:07.674023 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "8c9d6c30b988406f9347f51c8dcc0159"
format_stamp: "Formatted at 2026-09-23 20:46:07 on dist-test-slave-dn29"
server_key: "44de56ccf17be6ee525b2762714c8156"
server_key_iv: "62080f9f7fb86e549fb2a7a3f15cc6d7"
server_key_version: "encryptionkey@0"
I20260923 20:46:07.680298 6043 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:46:07.684355 6058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:07.685947 6043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:46:07.686141 6043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "8c9d6c30b988406f9347f51c8dcc0159"
format_stamp: "Formatted at 2026-09-23 20:46:07 on dist-test-slave-dn29"
server_key: "44de56ccf17be6ee525b2762714c8156"
server_key_iv: "62080f9f7fb86e549fb2a7a3f15cc6d7"
server_key_version: "encryptionkey@0"
I20260923 20:46:07.686391 6043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:07.711859 6043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:07.716296 6043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:07.716598 6043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:07.717656 6043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:07.719004 6043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:07.719080 6043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:07.719249 6043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:07.719317 6043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:07.755153 6043 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:37299
I20260923 20:46:07.755188 6170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:37299 every 8 connection(s)
I20260923 20:46:07.756548 6043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:07.758190 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6043
I20260923 20:46:07.758430 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:07.759191 5406 external_mini_cluster.cc:1428] Setting key 6ef47ce6db51ccc478710d485b66ab7c
Sep 23 20:46:07 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196367, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:07.778752 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:07.759638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:50099 (local address 127.5.71.190:40831)
0923 20:46:07.760124 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:07.760130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:07.761366 (+ 1236us) server_negotiation.cc:407] Connection header received
0923 20:46:07.763457 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:07.763466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:07.763562 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:07.763679 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:07.765847 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:07.766549 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:07.768303 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:07.768714 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:07.768908 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:07.772626 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:07.772631 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:07.772634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:07.772667 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:07.775295 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:07.776190 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:07.776200 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:07.776203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:07.776318 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:07.776930 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:07.776934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:07.776936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:07.777252 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:07.777399 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:07.778323 (+ 924us) server_negotiation.cc:299] Negotiation successful
0923 20:46:07.778496 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":269,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:07.779999 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:07.760296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40831 (local address 127.5.71.129:50099)
0923 20:46:07.761194 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:07.761242 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:07.762946 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:07.764058 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:07.764067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:07.764678 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:07.765543 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:07.765578 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:07.766717 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:07.766722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:07.768146 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:07.768158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:07.768998 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:07.769006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:07.769262 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:07.769974 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:07.769997 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:07.772380 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:07.775449 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:07.775459 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:07.775478 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:07.776055 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:07.776432 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:07.776437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:07.776441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:07.776636 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:07.777399 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:07.777406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:07.777771 (+ 365us) client_negotiation.cc:769] Sending connection context
0923 20:46:07.778176 (+ 405us) client_negotiation.cc:240] Negotiation successful
0923 20:46:07.778646 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":377,"threads_started":1}
I20260923 20:46:07.781666 6171 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40831
I20260923 20:46:07.782034 6171 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:07.782750 6171 heartbeater.cc:511] Master 127.5.71.190:40831 requested a full tablet report, sending...
I20260923 20:46:07.784909 5982 ts_manager.cc:194] Registered new tserver with Master: 8c9d6c30b988406f9347f51c8dcc0159 (127.5.71.129:37299)
I20260923 20:46:07.786336 5982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:50099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:07.804435 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40831
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:08.141240 6177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:08.141601 6177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:08.141793 6177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:08.149309 6177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:08.149487 6177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:08.149668 6177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:08.159948 6177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:08.161443 6177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:08.162904 6177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:08.177027 6182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:08.177309 6183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:08.178269 6185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:08.178295 6177 server_base.cc:1034] running on GCE node
I20260923 20:46:08.178993 6177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:08.180120 6177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:08.181413 6177 hybrid_clock.cc:648] HybridClock initialized: now 1790196368181362 us; error 42 us; skew 500 ppm
I20260923 20:46:08.181804 6177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:08 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196368, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:08.185355 6177 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:08.186911 6177 webserver.cc:460] Webserver started at http://127.5.71.130:35247/ using document root <none> and password file <none>
I20260923 20:46:08.187681 6177 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:08.187867 6177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:08.188215 6177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:08.191493 6177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "5908bec580d842f3bf380931f750f87c"
format_stamp: "Formatted at 2026-09-23 20:46:08 on dist-test-slave-dn29"
server_key: "00ecb7f11ff1f3cfde28f2382b0d20e3"
server_key_iv: "a405d98fe2ebcdfdf752d7808fb274cb"
server_key_version: "encryptionkey@0"
I20260923 20:46:08.192536 6177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "5908bec580d842f3bf380931f750f87c"
format_stamp: "Formatted at 2026-09-23 20:46:08 on dist-test-slave-dn29"
server_key: "00ecb7f11ff1f3cfde28f2382b0d20e3"
server_key_iv: "a405d98fe2ebcdfdf752d7808fb274cb"
server_key_version: "encryptionkey@0"
I20260923 20:46:08.197976 6177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260923 20:46:08.201889 6192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:08.203363 6177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:46:08.203552 6177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "5908bec580d842f3bf380931f750f87c"
format_stamp: "Formatted at 2026-09-23 20:46:08 on dist-test-slave-dn29"
server_key: "00ecb7f11ff1f3cfde28f2382b0d20e3"
server_key_iv: "a405d98fe2ebcdfdf752d7808fb274cb"
server_key_version: "encryptionkey@0"
I20260923 20:46:08.203763 6177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:08.233129 6177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:08.237938 6177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:08.238270 6177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:08.239609 6177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:08.241158 6177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:08.241235 6177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:08.241370 6177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:08.241421 6177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:08.277314 6177 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:44337
I20260923 20:46:08.277448 6304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:44337 every 8 connection(s)
I20260923 20:46:08.278687 6177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:08.283349 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6177
I20260923 20:46:08.283563 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:08.284096 5406 external_mini_cluster.cc:1428] Setting key 2ac69ddb35dbd9e5f402d81201270ac9
Sep 23 20:46:08 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196368, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:08.300920 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.281799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:51245 (local address 127.5.71.190:40831)
0923 20:46:08.282292 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:08.282296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:08.283479 (+ 1183us) server_negotiation.cc:407] Connection header received
0923 20:46:08.285524 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:08.285530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:08.285631 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:08.285756 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:08.287391 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.288318 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.289969 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.290406 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.290540 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.294723 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:08.294728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:08.294731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:08.294769 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:08.297680 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:08.298457 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:08.298463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:08.298467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:08.298597 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:08.299038 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:08.299044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:08.299047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:08.299340 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:08.299498 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:08.300297 (+ 799us) server_negotiation.cc:299] Negotiation successful
0923 20:46:08.300528 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":282,"threads_started":1}
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:08.302367 6308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.282372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40831 (local address 127.5.71.130:51245)
0923 20:46:08.283286 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:08.283339 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:08.285234 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:08.286031 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:08.286040 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:08.286570 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:08.287204 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.287229 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.288479 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.288484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.289819 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.289831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.290655 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.290662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.290991 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.291735 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:08.291759 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:08.294452 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:08.297852 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:08.297862 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:08.297883 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:08.298328 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:08.298712 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:08.298717 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:08.298721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:08.298920 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:08.299472 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:08.299481 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:08.299930 (+ 449us) client_negotiation.cc:769] Sending connection context
0923 20:46:08.300211 (+ 281us) client_negotiation.cc:240] Negotiation successful
0923 20:46:08.300835 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":324,"threads_started":1}
I20260923 20:46:08.304278 6305 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:08.304729 6305 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:08.305807 6305 heartbeater.cc:511] Master 127.5.71.190:40831 requested a full tablet report, sending...
I20260923 20:46:08.307739 5982 ts_manager.cc:194] Registered new tserver with Master: 5908bec580d842f3bf380931f750f87c (127.5.71.130:44337)
I20260923 20:46:08.309244 5982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:51245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:08.332878 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40831
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:08.666333 6311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:08.666633 6311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:08.666819 6311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:08.674327 6311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:08.674443 6311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:08.674674 6311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:08.684933 6311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40831
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:08.686271 6311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:08.687724 6311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:08.701403 6316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:08.702060 6317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:08.702446 6319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:08.703215 6311 server_base.cc:1034] running on GCE node
I20260923 20:46:08.704038 6311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:08.705267 6311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:08.706564 6311 hybrid_clock.cc:648] HybridClock initialized: now 1790196368706430 us; error 58 us; skew 500 ppm
I20260923 20:46:08.707082 6311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:08 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196368, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:08.710995 6311 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:08.712708 6311 webserver.cc:460] Webserver started at http://127.5.71.131:32993/ using document root <none> and password file <none>
I20260923 20:46:08.713445 6311 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:08.713554 6311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:08.713855 6311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:08.717099 6311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "c8a171de9386440e8e4015e7c0472acc"
format_stamp: "Formatted at 2026-09-23 20:46:08 on dist-test-slave-dn29"
server_key: "eed55fb7dedbfcfbac9fa457a0f4c8a2"
server_key_iv: "e61924fb3cdae86ca8034079477888bd"
server_key_version: "encryptionkey@0"
I20260923 20:46:08.717936 6311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "c8a171de9386440e8e4015e7c0472acc"
format_stamp: "Formatted at 2026-09-23 20:46:08 on dist-test-slave-dn29"
server_key: "eed55fb7dedbfcfbac9fa457a0f4c8a2"
server_key_iv: "e61924fb3cdae86ca8034079477888bd"
server_key_version: "encryptionkey@0"
I20260923 20:46:08.723781 6311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:46:08.727905 6326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:08.729333 6311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:08.729532 6311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "c8a171de9386440e8e4015e7c0472acc"
format_stamp: "Formatted at 2026-09-23 20:46:08 on dist-test-slave-dn29"
server_key: "eed55fb7dedbfcfbac9fa457a0f4c8a2"
server_key_iv: "e61924fb3cdae86ca8034079477888bd"
server_key_version: "encryptionkey@0"
I20260923 20:46:08.729734 6311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:08.745694 6311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:08.750360 6311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:08.750684 6311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:08.751729 6311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:08.753316 6311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:08.753428 6311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:08.753607 6311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:08.753682 6311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:08.789518 6171 heartbeater.cc:503] Master 127.5.71.190:40831 was elected leader, sending a full tablet report...
I20260923 20:46:08.792910 6311 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:42671
I20260923 20:46:08.793000 6438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:42671 every 8 connection(s)
I20260923 20:46:08.794327 6311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:08.802507 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6311
I20260923 20:46:08.802711 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:08.803139 5406 external_mini_cluster.cc:1428] Setting key c4ff759df4f1d6d186b58e7d8adee288
Sep 23 20:46:08 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196368, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:08.816956 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.797259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:45031 (local address 127.5.71.190:40831)
0923 20:46:08.797433 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:08.797439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:08.798632 (+ 1193us) server_negotiation.cc:407] Connection header received
0923 20:46:08.800555 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:08.800561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:08.800663 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:08.800775 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:08.802298 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.803340 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.805641 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.806357 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.806506 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.810765 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:08.810770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:08.810779 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:08.810825 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:08.813617 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:08.814475 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:08.814479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:08.814482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:08.814613 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:08.815120 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:08.815124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:08.815127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:08.815448 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:08.815574 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:08.816529 (+ 955us) server_negotiation.cc:299] Negotiation successful
0923 20:46:08.816710 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 20:46:08.818284 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.797744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40831 (local address 127.5.71.131:45031)
0923 20:46:08.798470 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:08.798520 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:08.800292 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:08.801006 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:08.801014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:08.801506 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:08.802077 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.802094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.803546 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.803552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.805466 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.805480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.806747 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.806756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.807126 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.807957 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:08.807982 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:08.810509 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:08.813831 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:08.813839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:08.813855 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:08.814320 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:08.814732 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:08.814736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:08.814738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:08.814970 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:08.815616 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:08.815623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:08.816116 (+ 493us) client_negotiation.cc:769] Sending connection context
0923 20:46:08.816383 (+ 267us) client_negotiation.cc:240] Negotiation successful
0923 20:46:08.816866 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":304,"threads_started":1}
I20260923 20:46:08.819868 6439 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40831
I20260923 20:46:08.820192 6439 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:08.821007 6439 heartbeater.cc:511] Master 127.5.71.190:40831 requested a full tablet report, sending...
I20260923 20:46:08.822477 5982 ts_manager.cc:194] Registered new tserver with Master: c8a171de9386440e8e4015e7c0472acc (127.5.71.131:42671)
I20260923 20:46:08.823308 5982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:45031
I20260923 20:46:08.832297 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:46:08.855270 6307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.841902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.5.71.190:40831)
0923 20:46:08.842085 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:08.842089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:08.842524 (+ 435us) server_negotiation.cc:407] Connection header received
0923 20:46:08.842743 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:08.842747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:08.842848 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:08.842959 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:08.844528 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.845405 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.846773 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.847167 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.847270 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.849801 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:08.849808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:08.849811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:08.849845 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:08.852503 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:08.853171 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:08.853178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:08.853181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:08.853317 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:08.853695 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:08.853700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:08.853703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:08.854036 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:08.854331 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:08.854722 (+ 391us) server_negotiation.cc:299] Negotiation successful
0923 20:46:08.854942 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:46:08.864185 5982 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:08.878171 5982 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:08.928136 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.902274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:37299 (local address 127.0.0.1:56506)
0923 20:46:08.915042 (+ 12768us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:08.915068 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:08.915266 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:08.917537 (+ 2271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:08.917545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:08.917584 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:08.918967 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.919011 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.922127 (+ 3116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.922132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.925243 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.925258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.926880 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.926887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.927134 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.927196 (+ 62us) client_negotiation.cc:769] Sending connection context
0923 20:46:08.927341 (+ 145us) client_negotiation.cc:240] Negotiation successful
0923 20:46:08.927549 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12566,"mutex_wait_us":268,"thread_start_us":12324,"threads_started":1}
I20260923 20:46:08.928869 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.902960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.5.71.129:37299)
0923 20:46:08.916620 (+ 13660us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:08.916631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:08.916682 (+ 51us) server_negotiation.cc:407] Connection header received
0923 20:46:08.916848 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:08.916857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:08.917140 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:08.918192 (+ 1052us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:08.919449 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.921954 (+ 2505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.925463 (+ 3509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.926624 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.926790 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.927058 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:08.928308 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0923 20:46:08.928505 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13287,"thread_start_us":12974,"threads_started":1}
I20260923 20:46:08.931567 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.901619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.5.71.131:42671)
0923 20:46:08.914684 (+ 13065us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:08.914703 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:08.914787 (+ 84us) server_negotiation.cc:407] Connection header received
0923 20:46:08.914960 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:08.914974 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:08.915460 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:08.917213 (+ 1753us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:08.919848 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.921875 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.927115 (+ 5240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.929915 (+ 2800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.930114 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.930442 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:08.931055 (+ 613us) server_negotiation.cc:299] Negotiation successful
0923 20:46:08.931236 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12680,"thread_start_us":12276,"threads_started":1}
I20260923 20:46:08.931716 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.900821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:44337 (local address 127.0.0.1:47240)
0923 20:46:08.908222 (+ 7401us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:08.908285 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:08.909711 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:08.917704 (+ 7993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:08.917713 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:08.917741 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:08.919449 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.919464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.922127 (+ 2663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.922132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.927412 (+ 5280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.927434 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.930442 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.930451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.930703 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.930748 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 20:46:08.930835 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 20:46:08.931011 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6873,"mutex_wait_us":92,"thread_start_us":522,"threads_started":1}
I20260923 20:46:08.932231 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.900999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:42671 (local address 127.0.0.1:52634)
0923 20:46:08.910699 (+ 9700us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:08.910736 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:08.910969 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:08.917575 (+ 6606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:08.917589 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:08.917659 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:08.919541 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.919554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.922122 (+ 2568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.922128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.926791 (+ 4663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:08.926811 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.930616 (+ 3805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.930623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:08.930833 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.930870 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:46:08.930970 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 20:46:08.931097 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9428,"mutex_wait_us":31,"thread_start_us":10136,"threads_started":1}
I20260923 20:46:08.932682 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:08.901035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.5.71.130:44337)
0923 20:46:08.917028 (+ 15993us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:08.917036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:08.917063 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:46:08.917151 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:08.917159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:08.917442 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:08.917714 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:08.919896 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.921993 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.927739 (+ 5746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:08.930155 (+ 2416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:08.931687 (+ 1532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:08.931982 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:08.932073 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 20:46:08.932301 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15813,"thread_start_us":15679,"threads_started":1}
I20260923 20:46:08.933122 6106 tablet_service.cc:1467] Processing CreateTablet for tablet 23587669e71547309d48d4e9a70921eb (DEFAULT_TABLE table=test-table [id=bd972543047e4128939e25334d0c897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:08.934721 6106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23587669e71547309d48d4e9a70921eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:08.936717 6374 tablet_service.cc:1467] Processing CreateTablet for tablet 23587669e71547309d48d4e9a70921eb (DEFAULT_TABLE table=test-table [id=bd972543047e4128939e25334d0c897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:08.936941 6240 tablet_service.cc:1467] Processing CreateTablet for tablet 23587669e71547309d48d4e9a70921eb (DEFAULT_TABLE table=test-table [id=bd972543047e4128939e25334d0c897e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:08.938230 6374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23587669e71547309d48d4e9a70921eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:08.938246 6240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23587669e71547309d48d4e9a70921eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:08.955003 6458 tablet_bootstrap.cc:492] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: Bootstrap starting.
I20260923 20:46:08.959401 6459 tablet_bootstrap.cc:492] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: Bootstrap starting.
I20260923 20:46:08.962173 6458 tablet_bootstrap.cc:654] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:08.965482 6457 tablet_bootstrap.cc:492] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: Bootstrap starting.
I20260923 20:46:08.966464 6459 tablet_bootstrap.cc:654] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:08.966681 6458 log.cc:824] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:08.968510 6459 log.cc:824] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:08.970983 6458 tablet_bootstrap.cc:492] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: No bootstrap required, opened a new log
I20260923 20:46:08.971012 6457 tablet_bootstrap.cc:654] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:08.971323 6458 ts_tablet_manager.cc:1397] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260923 20:46:08.972656 6459 tablet_bootstrap.cc:492] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: No bootstrap required, opened a new log
I20260923 20:46:08.972632 6457 log.cc:824] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:08.972963 6459 ts_tablet_manager.cc:1397] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260923 20:46:08.976882 6457 tablet_bootstrap.cc:492] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: No bootstrap required, opened a new log
I20260923 20:46:08.978070 6457 ts_tablet_manager.cc:1397] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.005s
I20260923 20:46:08.982360 6458 raft_consensus.cc:348] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:08.982918 6458 raft_consensus.cc:374] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:08.983101 6458 raft_consensus.cc:729] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a171de9386440e8e4015e7c0472acc, State: Initialized, Role: FOLLOWER
I20260923 20:46:08.983958 6458 consensus_queue.cc:260] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:08.985445 6439 heartbeater.cc:503] Master 127.5.71.190:40831 was elected leader, sending a full tablet report...
I20260923 20:46:08.986073 6458 ts_tablet_manager.cc:1428] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260923 20:46:08.990144 6459 raft_consensus.cc:348] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:08.990947 6459 raft_consensus.cc:374] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:08.991182 6459 raft_consensus.cc:729] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5908bec580d842f3bf380931f750f87c, State: Initialized, Role: FOLLOWER
I20260923 20:46:08.992368 6459 consensus_queue.cc:260] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:08.994264 6305 heartbeater.cc:503] Master 127.5.71.190:40831 was elected leader, sending a full tablet report...
I20260923 20:46:08.995307 6459 ts_tablet_manager.cc:1428] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: Time spent starting tablet: real 0.022s user 0.018s sys 0.004s
I20260923 20:46:08.998006 6457 raft_consensus.cc:348] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:08.998764 6457 raft_consensus.cc:374] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:08.998973 6457 raft_consensus.cc:729] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9d6c30b988406f9347f51c8dcc0159, State: Initialized, Role: FOLLOWER
I20260923 20:46:09.000008 6457 consensus_queue.cc:260] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:09.002468 6457 ts_tablet_manager.cc:1428] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: Time spent starting tablet: real 0.024s user 0.017s sys 0.003s
W20260923 20:46:09.009049 6172 tablet.cc:2369] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:09.031070 6306 tablet.cc:2369] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:09.046311 6440 tablet.cc:2369] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:09.123667 6465 raft_consensus.cc:482] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:09.124284 6465 raft_consensus.cc:504] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:09.126734 6465 leader_election.cc:288] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5908bec580d842f3bf380931f750f87c (127.5.71.130:44337), c8a171de9386440e8e4015e7c0472acc (127.5.71.131:42671)
I20260923 20:46:09.140017 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.127540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:38547 (local address 127.5.71.131:42671)
0923 20:46:09.127841 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.127849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.129438 (+ 1589us) server_negotiation.cc:407] Connection header received
0923 20:46:09.129808 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.129818 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.129918 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.130031 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:09.131407 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.132991 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.136977 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.138503 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.138802 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.138920 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.139327 (+ 407us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.139469 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:46:09.140327 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.127037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:42671 (local address 127.5.71.129:38547)
0923 20:46:09.129261 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:09.129286 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:09.129534 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:09.130312 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:09.130316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:09.130344 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:09.131230 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:09.131242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.133138 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.133144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:09.136730 (+ 3586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:09.136744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.138886 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.138894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:09.139121 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.139170 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 20:46:09.139297 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 20:46:09.139468 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1976,"mutex_wait_us":38,"thread_start_us":1449,"threads_started":1}
I20260923 20:46:09.141588 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.127691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:44337 (local address 127.5.71.129:59689)
0923 20:46:09.130337 (+ 2646us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:09.130354 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:09.130504 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:09.130967 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:09.130971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:09.130995 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:09.131933 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:09.131944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.133780 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.133789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:09.136991 (+ 3202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:09.137006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.140172 (+ 3166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.140182 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:09.140429 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.140466 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:46:09.140773 (+ 307us) client_negotiation.cc:240] Negotiation successful
0923 20:46:09.140920 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2461,"mutex_wait_us":1,"thread_start_us":2166,"threads_started":1}
I20260923 20:46:09.141698 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23587669e71547309d48d4e9a70921eb" candidate_uuid: "8c9d6c30b988406f9347f51c8dcc0159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a171de9386440e8e4015e7c0472acc" is_pre_election: true
I20260923 20:46:09.142503 6394 raft_consensus.cc:2454] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9d6c30b988406f9347f51c8dcc0159 in term 0.
I20260923 20:46:09.143330 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.127521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:59689 (local address 127.5.71.130:44337)
0923 20:46:09.127785 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.127815 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.130478 (+ 2663us) server_negotiation.cc:407] Connection header received
0923 20:46:09.130697 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.130712 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.130814 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.130951 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:09.132169 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.133570 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.137244 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.139411 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.139579 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.139679 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.140848 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.143025 (+ 2177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 20:46:09.143461 6060 leader_election.cc:302] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c9d6c30b988406f9347f51c8dcc0159, c8a171de9386440e8e4015e7c0472acc; no voters:
I20260923 20:46:09.144137 6465 raft_consensus.cc:2781] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:09.144392 6465 raft_consensus.cc:482] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:09.144611 6465 raft_consensus.cc:3037] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:09.144888 6260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23587669e71547309d48d4e9a70921eb" candidate_uuid: "8c9d6c30b988406f9347f51c8dcc0159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5908bec580d842f3bf380931f750f87c" is_pre_election: true
I20260923 20:46:09.145776 6260 raft_consensus.cc:2454] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9d6c30b988406f9347f51c8dcc0159 in term 0.
I20260923 20:46:09.148612 6465 raft_consensus.cc:504] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:09.149516 6465 leader_election.cc:288] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [CANDIDATE]: Term 1 election: Requested vote from peers 5908bec580d842f3bf380931f750f87c (127.5.71.130:44337), c8a171de9386440e8e4015e7c0472acc (127.5.71.131:42671)
I20260923 20:46:09.150106 6260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23587669e71547309d48d4e9a70921eb" candidate_uuid: "8c9d6c30b988406f9347f51c8dcc0159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5908bec580d842f3bf380931f750f87c"
I20260923 20:46:09.150256 6394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23587669e71547309d48d4e9a70921eb" candidate_uuid: "8c9d6c30b988406f9347f51c8dcc0159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8a171de9386440e8e4015e7c0472acc"
I20260923 20:46:09.150372 6260 raft_consensus.cc:3037] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:09.150552 6394 raft_consensus.cc:3037] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:09.154495 6260 raft_consensus.cc:2454] T 23587669e71547309d48d4e9a70921eb P 5908bec580d842f3bf380931f750f87c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9d6c30b988406f9347f51c8dcc0159 in term 1.
I20260923 20:46:09.155058 6062 leader_election.cc:302] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5908bec580d842f3bf380931f750f87c, 8c9d6c30b988406f9347f51c8dcc0159; no voters:
I20260923 20:46:09.155489 6465 raft_consensus.cc:2781] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:09.155754 6394 raft_consensus.cc:2454] T 23587669e71547309d48d4e9a70921eb P c8a171de9386440e8e4015e7c0472acc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9d6c30b988406f9347f51c8dcc0159 in term 1.
I20260923 20:46:09.156587 6465 raft_consensus.cc:686] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9d6c30b988406f9347f51c8dcc0159, State: Running, Role: LEADER
I20260923 20:46:09.157269 6465 consensus_queue.cc:237] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } }
I20260923 20:46:09.167330 5982 catalog_manager.cc:5495] T 23587669e71547309d48d4e9a70921eb P 8c9d6c30b988406f9347f51c8dcc0159 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c9d6c30b988406f9347f51c8dcc0159 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "8c9d6c30b988406f9347f51c8dcc0159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5908bec580d842f3bf380931f750f87c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 44337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8a171de9386440e8e4015e7c0472acc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c9d6c30b988406f9347f51c8dcc0159" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37299 } health_report { overall_health: HEALTHY } } }
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:09.237962 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.214309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.5.71.129:37299)
0923 20:46:09.214613 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.214618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.215703 (+ 1085us) server_negotiation.cc:407] Connection header received
0923 20:46:09.216118 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.216127 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.216279 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.216403 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:09.218977 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.220105 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.221859 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.222526 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.222654 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.229049 (+ 6395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:09.229066 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:09.229085 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:09.229129 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:09.233403 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.234389 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.234403 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.234409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.234930 (+ 521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.235456 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.235460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.235462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.236171 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:09.236500 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.237033 (+ 533us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.237483 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196369, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:09.298022 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.274036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56518 (local address 127.5.71.129:37299)
0923 20:46:09.274276 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.274282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.274991 (+ 709us) server_negotiation.cc:407] Connection header received
0923 20:46:09.275281 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.275286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.275357 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.275479 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:09.278604 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.279682 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.282500 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.283268 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.283380 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.290817 (+ 7437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:09.290823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:09.290828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:09.290870 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:09.294142 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.295247 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.295253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.295255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.295394 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.296018 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.296022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.296025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.296345 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:09.296760 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.297070 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.297422 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:09.355115 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.336094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.5.71.129:37299)
0923 20:46:09.336411 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.336427 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.336959 (+ 532us) server_negotiation.cc:407] Connection header received
0923 20:46:09.337199 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.337205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.337319 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.337444 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:09.339473 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.340562 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.342267 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.342953 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.343104 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.348790 (+ 5686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:09.348797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:09.348800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:09.348841 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:09.352387 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.353172 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.353175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.353178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.353373 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.353709 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.353712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.353714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.354043 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:09.354306 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.354523 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.354682 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260923 20:46:09.357550 6086 tablet_service.cc:2908] Rejecting scan request for tablet 23587669e71547309d48d4e9a70921eb: Uninitialized: clean time has not yet been initialized
I20260923 20:46:09.362344 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6043
I20260923 20:46:09.377120 6456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.363270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.5.71.129:37299)
0923 20:46:09.363471 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.363477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.363901 (+ 424us) server_negotiation.cc:407] Connection header received
0923 20:46:09.364080 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.364083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.364148 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.364236 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:09.365886 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.366619 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.368193 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.368565 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.368667 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.371556 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:09.371561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:09.371564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:09.371594 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:09.374192 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.375024 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.375028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.375030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.375149 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.375538 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.375541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.375543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.375884 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:09.376067 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.376335 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.376598 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 20:46:09.378074 6166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:09.474187 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6043
I20260923 20:46:09.493541 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6177
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:09.511412 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.494615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.5.71.130:44337)
0923 20:46:09.494844 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.494849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.494870 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:46:09.494925 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.494929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.494996 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.495082 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:09.496683 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.497508 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.498927 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.499335 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.499438 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.504111 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:09.504125 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:09.504146 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:09.504202 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:09.508344 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.509119 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.509128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.509131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.509273 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.509664 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.509672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.509707 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.510091 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:09.510436 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.510781 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.511013 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260923 20:46:09.512365 6300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:09.603498 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6177
I20260923 20:46:09.620765 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6311
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[5937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:09.637393 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:09.621610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.5.71.131:42671)
0923 20:46:09.621791 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:09.621796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:09.621816 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:46:09.621922 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:09.621926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:09.621992 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:09.622089 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:09.623493 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.624402 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.625809 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:09.626223 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:09.626346 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:09.630691 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:09.630709 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:09.630729 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:09.630773 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:09.634095 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.635062 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.635071 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.635073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.635255 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:09.635726 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:09.635731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:09.635733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:09.636297 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:09.636551 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:09.636896 (+ 345us) server_negotiation.cc:299] Negotiation successful
0923 20:46:09.637073 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:46:09.638172 6434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:09.737425 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6311
I20260923 20:46:09.754208 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 5950
I20260923 20:46:09.755609 6012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:09.865087 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 5950
2026-09-23T20:46:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4573 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[6487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[6487](info): set up 2 sockets
Sep 23 20:46:09 dist-test-slave-dn29 krb5kdc[6487](info): commencing operation
krb5kdc: starting...
W20260923 20:46:10.960470 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:11 dist-test-slave-dn29 krb5kdc[6487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:11.164947 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38467
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:38467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:11.509444 6500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:11.509995 6500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:11.510192 6500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:11.519701 6500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:11.519941 6500 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 20:46:11.520020 6500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:11.520123 6500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:11.520213 6500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:11.520300 6500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:11.530622 6500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:38467
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:11.532173 6500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:11.533564 6500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:11.545909 6508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:11.546002 6505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:11.545966 6506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:11.547163 6500 server_base.cc:1034] running on GCE node
I20260923 20:46:11.548163 6500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:11.549613 6500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:11.550845 6500 hybrid_clock.cc:648] HybridClock initialized: now 1790196371550751 us; error 55 us; skew 500 ppm
I20260923 20:46:11.551362 6500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:11 dist-test-slave-dn29 krb5kdc[6487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196371, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:11.554993 6500 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:11.556619 6500 webserver.cc:460] Webserver started at http://127.5.71.190:34105/ using document root <none> and password file <none>
I20260923 20:46:11.557386 6500 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:11.557477 6500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:11.557870 6500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:11.561196 6500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "1b9120c605e344a2a0a6135d93c0bd02"
format_stamp: "Formatted at 2026-09-23 20:46:11 on dist-test-slave-dn29"
server_key: "4d73d0e6580156540022a8725ab384da"
server_key_iv: "ff288e6086395d912a0d7c799a5b8115"
server_key_version: "encryptionkey@0"
I20260923 20:46:11.562086 6500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "1b9120c605e344a2a0a6135d93c0bd02"
format_stamp: "Formatted at 2026-09-23 20:46:11 on dist-test-slave-dn29"
server_key: "4d73d0e6580156540022a8725ab384da"
server_key_iv: "ff288e6086395d912a0d7c799a5b8115"
server_key_version: "encryptionkey@0"
I20260923 20:46:11.567864 6500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:46:11.572098 6515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:11.574091 6500 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260923 20:46:11.574329 6500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "1b9120c605e344a2a0a6135d93c0bd02"
format_stamp: "Formatted at 2026-09-23 20:46:11 on dist-test-slave-dn29"
server_key: "4d73d0e6580156540022a8725ab384da"
server_key_iv: "ff288e6086395d912a0d7c799a5b8115"
server_key_version: "encryptionkey@0"
I20260923 20:46:11.574581 6500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:11.588963 6500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:11.593643 6500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:11.593935 6500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:11.617370 6500 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:38467
I20260923 20:46:11.617378 6576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:38467 every 8 connection(s)
I20260923 20:46:11.618834 6500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:11.623103 6577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:11.624060 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6500
I20260923 20:46:11.624296 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:11.624958 5406 external_mini_cluster.cc:1428] Setting key 6759facc722b7c7e2a0882587099aef0
I20260923 20:46:11.639060 6577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: Bootstrap starting.
Sep 23 20:46:11 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:11.642731 6577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:11.644315 6577 log.cc:824] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:11.647998 6577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: No bootstrap required, opened a new log
I20260923 20:46:11.651155 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:11.627047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.5.71.190:38467)
0923 20:46:11.627964 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:11.627981 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:11.628037 (+ 56us) server_negotiation.cc:407] Connection header received
0923 20:46:11.630407 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:11.630443 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:11.631042 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:11.631653 (+ 611us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:11.633087 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:11.635325 (+ 2238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:11.636998 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:11.638163 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:11.638381 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:11.642453 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:11.642468 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:11.642493 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:11.642551 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:11.646699 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:11.647490 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:11.647500 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:11.647509 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:11.647751 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:11.648193 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:11.648197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:11.648199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:11.648834 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:11.649194 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:11.649657 (+ 463us) server_negotiation.cc:299] Negotiation successful
0923 20:46:11.650106 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":348,"threads_started":1}
I20260923 20:46:11.657238 6577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9120c605e344a2a0a6135d93c0bd02" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38467 } }
I20260923 20:46:11.657730 6577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:11.657841 6577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9120c605e344a2a0a6135d93c0bd02, State: Initialized, Role: FOLLOWER
I20260923 20:46:11.658385 6577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9120c605e344a2a0a6135d93c0bd02" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38467 } }
I20260923 20:46:11.658605 6577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:11.658710 6577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:11.658855 6577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:11.661237 6577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9120c605e344a2a0a6135d93c0bd02" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38467 } }
I20260923 20:46:11.661763 6577 leader_election.cc:302] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b9120c605e344a2a0a6135d93c0bd02; no voters:
I20260923 20:46:11.662423 6577 leader_election.cc:288] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:11.662601 6582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:11.664269 6582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [term 1 LEADER]: Becoming Leader. State: Replica: 1b9120c605e344a2a0a6135d93c0bd02, State: Running, Role: LEADER
I20260923 20:46:11.664848 6582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9120c605e344a2a0a6135d93c0bd02" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38467 } }
I20260923 20:46:11.665089 6577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:11.668376 6584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b9120c605e344a2a0a6135d93c0bd02. Latest consensus state: current_term: 1 leader_uuid: "1b9120c605e344a2a0a6135d93c0bd02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9120c605e344a2a0a6135d93c0bd02" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38467 } } }
I20260923 20:46:11.668823 6583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b9120c605e344a2a0a6135d93c0bd02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9120c605e344a2a0a6135d93c0bd02" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38467 } } }
I20260923 20:46:11.669077 6584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:11.669212 6583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:11.670166 6591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:11.677950 6591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:11.688511 6591 catalog_manager.cc:1347] Generated new cluster ID: 2cd5efb833a14e66a0dad26b757b3ffb
I20260923 20:46:11.688766 6591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:11.714046 6591 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:11.715619 6591 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:11.727185 6591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: Generated new TSK 0
I20260923 20:46:11.728228 6591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:11.779597 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38467
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:12.120092 6603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:12.120422 6603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:12.120523 6603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:12.127681 6603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:12.127863 6603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:12.127985 6603 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:46:12.128072 6603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:12.138578 6603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:38467
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:12.140038 6603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:12.141530 6603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:12.155498 6608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:12.155551 6609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:12.155572 6611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:12.156306 6603 server_base.cc:1034] running on GCE node
I20260923 20:46:12.156863 6603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:12.157948 6603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:12.159209 6603 hybrid_clock.cc:648] HybridClock initialized: now 1790196372159113 us; error 66 us; skew 500 ppm
I20260923 20:46:12.159562 6603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:12 dist-test-slave-dn29 krb5kdc[6487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196372, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:12.162981 6603 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:12.164577 6603 webserver.cc:460] Webserver started at http://127.5.71.129:34579/ using document root <none> and password file <none>
I20260923 20:46:12.165393 6603 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:12.165482 6603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:12.165741 6603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:12.169054 6603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "67945d63a7a84b4fb01325688202d30a"
format_stamp: "Formatted at 2026-09-23 20:46:12 on dist-test-slave-dn29"
server_key: "2af038fcf8278d3a8024d20ac2e0d69c"
server_key_iv: "8107aa546db39fb0f6d125b047cf63f6"
server_key_version: "encryptionkey@0"
I20260923 20:46:12.170086 6603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "67945d63a7a84b4fb01325688202d30a"
format_stamp: "Formatted at 2026-09-23 20:46:12 on dist-test-slave-dn29"
server_key: "2af038fcf8278d3a8024d20ac2e0d69c"
server_key_iv: "8107aa546db39fb0f6d125b047cf63f6"
server_key_version: "encryptionkey@0"
I20260923 20:46:12.175732 6603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:12.179955 6618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:12.181535 6603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:12.181819 6603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "67945d63a7a84b4fb01325688202d30a"
format_stamp: "Formatted at 2026-09-23 20:46:12 on dist-test-slave-dn29"
server_key: "2af038fcf8278d3a8024d20ac2e0d69c"
server_key_iv: "8107aa546db39fb0f6d125b047cf63f6"
server_key_version: "encryptionkey@0"
I20260923 20:46:12.182109 6603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:12.198164 6603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:12.202849 6603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:12.203143 6603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:12.206350 6603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:12.206475 6603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:12.206620 6603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:12.206715 6603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:46:12 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196372, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:12.231518 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.207079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:50795 (local address 127.5.71.190:38467)
0923 20:46:12.209660 (+ 2581us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:12.209666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:12.209695 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:46:12.211191 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:12.211205 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:12.211299 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:12.211437 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:12.213790 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.215133 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.217696 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.218476 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.218731 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.223327 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:12.223335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:12.223340 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:12.223388 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:12.227222 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.228319 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.228324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.228328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.228504 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.229191 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.229205 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.229211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.229543 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:12.229741 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:12.230619 (+ 878us) server_negotiation.cc:299] Negotiation successful
0923 20:46:12.230944 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2442,"thread_start_us":2335,"threads_started":1}
I20260923 20:46:12.232280 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.207649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.5.71.129:50795)
0923 20:46:12.208797 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:12.208861 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:12.210899 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:12.211735 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:12.211756 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:12.212667 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:12.213560 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.213594 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.215333 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.215340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.217538 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.217550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.218920 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.218944 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.219217 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.220006 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:12.220047 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:12.223039 (+ 2992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:12.227428 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.227444 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.227465 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.228175 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.228649 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.228657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.228661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.229029 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.229811 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:12.229818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:12.230167 (+ 349us) client_negotiation.cc:769] Sending connection context
0923 20:46:12.230428 (+ 261us) client_negotiation.cc:240] Negotiation successful
0923 20:46:12.230840 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":403,"threads_started":1}
I20260923 20:46:12.256232 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.239361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.5.71.129:57379)
0923 20:46:12.240243 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:12.240260 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:12.240469 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:12.240864 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:12.240869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:12.241647 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:12.242233 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.242255 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.243848 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.243859 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.245934 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.245951 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.246804 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.246812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.246983 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.247893 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:12.247929 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:12.249279 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:12.253265 (+ 3986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.253270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.253278 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.253725 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.254138 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.254142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.254146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.254282 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.254955 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:12.254960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:12.255150 (+ 190us) client_negotiation.cc:769] Sending connection context
0923 20:46:12.255328 (+ 178us) client_negotiation.cc:240] Negotiation successful
0923 20:46:12.255724 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":569,"threads_started":1}
I20260923 20:46:12.256695 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.239321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:57379 (local address 127.5.71.190:38467)
0923 20:46:12.239514 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:12.239518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:12.240372 (+ 854us) server_negotiation.cc:407] Connection header received
0923 20:46:12.240626 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:12.240636 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:12.240703 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:12.240825 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:12.242468 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.243567 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.246102 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.246574 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.246731 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.249498 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:12.249508 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:12.249517 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:12.249552 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:12.253042 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.253862 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.253869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.253872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.254012 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.254404 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.254409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.254413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.254785 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:12.255943 (+ 1158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:12.256093 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 20:46:12.256362 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 20:46:12.267190 6603 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:40735
I20260923 20:46:12.267215 6738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:40735 every 8 connection(s)
I20260923 20:46:12.268532 6603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:12.271126 6739 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38467
I20260923 20:46:12.271263 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6603
I20260923 20:46:12.271427 6739 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:12.271494 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:12.272363 5406 external_mini_cluster.cc:1428] Setting key 00da12d6d20da710aa0ef820e8cafcb6
I20260923 20:46:12.272363 6739 heartbeater.cc:511] Master 127.5.71.190:38467 requested a full tablet report, sending...
I20260923 20:46:12.274730 6531 ts_manager.cc:194] Registered new tserver with Master: 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735)
I20260923 20:46:12.276275 6531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:50795
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:12.316160 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38467
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:12.658790 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:12.659094 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:12.659263 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:12.666885 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:12.667001 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:12.667140 6743 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:46:12.667235 6743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:12.677405 6743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:38467
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:12.678937 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:12.680562 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:12.695525 6749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:12.695525 6748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:12.696133 6751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:12.696939 6743 server_base.cc:1034] running on GCE node
I20260923 20:46:12.697710 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:12.698884 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:12.700119 6743 hybrid_clock.cc:648] HybridClock initialized: now 1790196372700057 us; error 36 us; skew 500 ppm
I20260923 20:46:12.700493 6743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:12 dist-test-slave-dn29 krb5kdc[6487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196372, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:12.703977 6743 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:12.705787 6743 webserver.cc:460] Webserver started at http://127.5.71.130:38181/ using document root <none> and password file <none>
I20260923 20:46:12.706804 6743 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:12.706950 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:12.707372 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:12.712339 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "372c8b3b589c477ca014990f930b36bb"
format_stamp: "Formatted at 2026-09-23 20:46:12 on dist-test-slave-dn29"
server_key: "c853e366830783f6cd889571f7eedeb7"
server_key_iv: "f065c95a3e90a94594e7cddb98225455"
server_key_version: "encryptionkey@0"
I20260923 20:46:12.713562 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "372c8b3b589c477ca014990f930b36bb"
format_stamp: "Formatted at 2026-09-23 20:46:12 on dist-test-slave-dn29"
server_key: "c853e366830783f6cd889571f7eedeb7"
server_key_iv: "f065c95a3e90a94594e7cddb98225455"
server_key_version: "encryptionkey@0"
I20260923 20:46:12.719928 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260923 20:46:12.723960 6758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:12.725526 6743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:12.725667 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "372c8b3b589c477ca014990f930b36bb"
format_stamp: "Formatted at 2026-09-23 20:46:12 on dist-test-slave-dn29"
server_key: "c853e366830783f6cd889571f7eedeb7"
server_key_iv: "f065c95a3e90a94594e7cddb98225455"
server_key_version: "encryptionkey@0"
I20260923 20:46:12.725865 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:12.743613 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:12.748872 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:12.749130 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:12.752367 6743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:12.752519 6743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:12.752701 6743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:12.752779 6743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:46:12 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196372, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:12.779352 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.753368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:34989 (local address 127.5.71.190:38467)
0923 20:46:12.753580 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:12.753587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:12.755401 (+ 1814us) server_negotiation.cc:407] Connection header received
0923 20:46:12.757545 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:12.757553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:12.757644 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:12.757777 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:12.760035 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.761509 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.763670 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.764643 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.764826 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.770678 (+ 5852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:12.770688 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:12.770695 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:12.770757 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:12.774668 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.775611 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.775615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.775618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.775737 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.776519 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.776528 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.776535 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.776996 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:12.777218 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:12.778392 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0923 20:46:12.778745 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 20:46:12.780165 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.754081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.5.71.130:34989)
0923 20:46:12.755202 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:12.755276 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:12.757236 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:12.758020 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:12.758028 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:12.758834 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:12.759768 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.759825 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.761731 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.761742 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.763498 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.763511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.764943 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.764953 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.765332 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.766311 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:12.766337 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:12.770229 (+ 3892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:12.774861 (+ 4632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.774872 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.774893 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.775464 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.775936 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.775941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.775948 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.776362 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.777180 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:12.777193 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:12.777742 (+ 549us) client_negotiation.cc:769] Sending connection context
0923 20:46:12.778181 (+ 439us) client_negotiation.cc:240] Negotiation successful
0923 20:46:12.778686 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":419,"threads_started":1}
I20260923 20:46:12.800292 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.786397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:39577 (local address 127.5.71.190:38467)
0923 20:46:12.786573 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:12.786577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:12.787375 (+ 798us) server_negotiation.cc:407] Connection header received
0923 20:46:12.787598 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:12.787604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:12.787673 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:12.787773 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:12.788734 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.789466 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.790745 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.791170 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.791318 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.793488 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:12.793493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:12.793496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:12.793536 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:12.796938 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.797874 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.797882 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.797885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.798045 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:12.798608 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:12.798614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:12.798620 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:12.799008 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:12.799176 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:12.799875 (+ 699us) server_negotiation.cc:299] Negotiation successful
0923 20:46:12.800057 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 20:46:12.801013 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:12.786500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.5.71.130:39577)
0923 20:46:12.787233 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:12.787249 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:12.787454 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:12.787831 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:12.787836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:12.788288 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:12.788578 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.788589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.789594 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.789601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.790598 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:12.790609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:12.791366 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:12.791373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:12.791564 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:12.792282 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:12.792299 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:12.793297 (+ 998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:12.797102 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.797107 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.797111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.797735 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.798212 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:12.798223 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:12.798231 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:12.798466 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:12.799157 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:12.799164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:12.799495 (+ 331us) client_negotiation.cc:769] Sending connection context
0923 20:46:12.799727 (+ 232us) client_negotiation.cc:240] Negotiation successful
0923 20:46:12.800287 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":460,"threads_started":1}
I20260923 20:46:12.813325 6743 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:36645
I20260923 20:46:12.813344 6877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:36645 every 8 connection(s)
I20260923 20:46:12.814643 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:12.816205 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6743
I20260923 20:46:12.816431 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:12.817039 5406 external_mini_cluster.cc:1428] Setting key e279c94ca92da9dce7a2bf5bddc4f49d
I20260923 20:46:12.817065 6878 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38467
I20260923 20:46:12.817322 6878 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:12.818010 6878 heartbeater.cc:511] Master 127.5.71.190:38467 requested a full tablet report, sending...
I20260923 20:46:12.819444 6531 ts_manager.cc:194] Registered new tserver with Master: 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645)
I20260923 20:46:12.820298 6531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:34989
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:12.864179 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38467
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:13.196201 6882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:13.196514 6882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:13.196696 6882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:13.203984 6882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:13.204100 6882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:13.204303 6882 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:46:13.204402 6882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:13.214393 6882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:38467
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:13.215821 6882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:13.217301 6882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:13.230672 6887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:13.230800 6888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:13.230938 6890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:13.231900 6882 server_base.cc:1034] running on GCE node
I20260923 20:46:13.232551 6882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:13.233538 6882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:13.234853 6882 hybrid_clock.cc:648] HybridClock initialized: now 1790196373234738 us; error 45 us; skew 500 ppm
I20260923 20:46:13.235231 6882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:13 dist-test-slave-dn29 krb5kdc[6487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:13.238620 6882 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:13.240401 6882 webserver.cc:460] Webserver started at http://127.5.71.131:33993/ using document root <none> and password file <none>
I20260923 20:46:13.241149 6882 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:13.241281 6882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:13.241604 6882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:13.244599 6882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8"
format_stamp: "Formatted at 2026-09-23 20:46:13 on dist-test-slave-dn29"
server_key: "f332f57a7cc86786c7bbc82bfc5e9974"
server_key_iv: "cb29918e6b6b9ad12c4a8b5eab75725b"
server_key_version: "encryptionkey@0"
I20260923 20:46:13.245422 6882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8"
format_stamp: "Formatted at 2026-09-23 20:46:13 on dist-test-slave-dn29"
server_key: "f332f57a7cc86786c7bbc82bfc5e9974"
server_key_iv: "cb29918e6b6b9ad12c4a8b5eab75725b"
server_key_version: "encryptionkey@0"
I20260923 20:46:13.250854 6882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:13.254876 6897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:13.256628 6882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260923 20:46:13.256845 6882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8"
format_stamp: "Formatted at 2026-09-23 20:46:13 on dist-test-slave-dn29"
server_key: "f332f57a7cc86786c7bbc82bfc5e9974"
server_key_iv: "cb29918e6b6b9ad12c4a8b5eab75725b"
server_key_version: "encryptionkey@0"
I20260923 20:46:13.257102 6882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:13.273480 6882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:13.278206 6882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:13.278522 6882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:13.279942 6739 heartbeater.cc:503] Master 127.5.71.190:38467 was elected leader, sending a full tablet report...
I20260923 20:46:13.282068 6882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:13.282212 6882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:13.282414 6882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:13.282497 6882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:46:13 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:13.308142 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.283012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:58147 (local address 127.5.71.190:38467)
0923 20:46:13.283199 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.283206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.285020 (+ 1814us) server_negotiation.cc:407] Connection header received
0923 20:46:13.288220 (+ 3200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.288226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.288313 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.288428 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:13.291132 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.292001 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.294985 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.295604 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.295755 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.300706 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:13.300716 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:13.300722 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:13.300787 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:13.304372 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:13.305113 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:13.305119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:13.305123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:13.305271 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:13.305707 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:13.305714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:13.305719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:13.305999 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:13.306157 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.307194 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.307544 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:46:13.308795 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.283672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.5.71.131:58147)
0923 20:46:13.284819 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.284871 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.287762 (+ 2891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.288788 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.288806 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.289806 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:13.290881 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.290914 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.292192 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.292196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.294743 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.294764 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.295883 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.295891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.296142 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.297113 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:13.297157 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:13.300451 (+ 3294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:13.304517 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:13.304526 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:13.304554 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:13.304986 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:13.305396 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:13.305400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:13.305403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:13.305586 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:13.306130 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:13.306138 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:13.306579 (+ 441us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.306912 (+ 333us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.307312 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":510,"threads_started":1}
I20260923 20:46:13.333755 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.316883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.5.71.131:36295)
0923 20:46:13.317758 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.317778 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.317918 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.318397 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.318404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.319056 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:13.319536 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.319549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.320867 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.320873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.322510 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.322532 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.323846 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.323856 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.324255 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.325011 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:13.325043 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:13.326181 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:13.330691 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:13.330700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:13.330706 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:13.331244 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:13.331843 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:13.331848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:13.331851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:13.331982 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:13.332838 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:13.332841 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:13.332994 (+ 153us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.333102 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.333421 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":565,"threads_started":1}
I20260923 20:46:13.334193 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.316868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:36295 (local address 127.5.71.190:38467)
0923 20:46:13.317129 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.317140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.317890 (+ 750us) server_negotiation.cc:407] Connection header received
0923 20:46:13.318051 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.318060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.318166 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.318307 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:13.319715 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.320718 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.322722 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.323573 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.323745 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.326409 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:13.326415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:13.326419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:13.326458 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:13.330487 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:13.331410 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:13.331420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:13.331427 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:13.331642 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:13.332109 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:13.332118 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:13.332124 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:13.332651 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:13.332861 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.333290 (+ 429us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.333607 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 20:46:13.344964 6882 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:46403
I20260923 20:46:13.344991 7016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:46403 every 8 connection(s)
I20260923 20:46:13.346347 6882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:13.348927 7017 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38467
I20260923 20:46:13.349149 7017 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:13.349997 7017 heartbeater.cc:511] Master 127.5.71.190:38467 requested a full tablet report, sending...
I20260923 20:46:13.351487 6531 ts_manager.cc:194] Registered new tserver with Master: b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131:46403)
I20260923 20:46:13.352488 6531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:58147
I20260923 20:46:13.353987 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 6882
I20260923 20:46:13.354200 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:13.354699 5406 external_mini_cluster.cc:1428] Setting key d918df5056e24daced91e201d674b35e
I20260923 20:46:13.358233 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 20:46:13 dist-test-slave-dn29 krb5kdc[6487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 20:46:13 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:13.408727 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.392765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.5.71.190:38467)
0923 20:46:13.393048 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.393058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.393304 (+ 246us) server_negotiation.cc:407] Connection header received
0923 20:46:13.393501 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.393506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.393573 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.393736 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:13.395190 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.396182 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.397565 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.398089 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.398212 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.402615 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:13.402622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:13.402626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:13.402668 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:13.405835 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:13.406622 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:13.406627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:13.406629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:13.406797 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:13.407154 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:13.407158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:13.407160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:13.407479 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:13.407675 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.408071 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.408295 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260923 20:46:13.413527 6531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:13.420697 6531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:13.448006 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.435814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40735 (local address 127.0.0.1:51142)
0923 20:46:13.436365 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.436395 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.436554 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.437721 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.437727 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.437768 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:13.438248 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.438260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.440915 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.440924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.443703 (+ 2779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.443727 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.446456 (+ 2729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.446462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.446709 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.446843 (+ 134us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.446985 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.447289 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":42,"thread_start_us":255,"threads_started":1}
I20260923 20:46:13.448849 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.434413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:36645 (local address 127.0.0.1:59478)
0923 20:46:13.435240 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.435298 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.435465 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.436591 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.436600 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.436636 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:13.437149 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.437167 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.440019 (+ 2852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.440028 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.443724 (+ 3696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.443746 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.446673 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.446683 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.446997 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.447066 (+ 69us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.447167 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.447388 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":16,"thread_start_us":348,"threads_started":1}
I20260923 20:46:13.449082 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.434619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.5.71.130:36645)
0923 20:46:13.435833 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.435842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.435874 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:46:13.436003 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.436014 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.436332 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.437449 (+ 1117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:13.437725 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.439771 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.444154 (+ 4383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.446331 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.448020 (+ 1689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.448331 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.448443 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.448698 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":376,"threads_started":1}
I20260923 20:46:13.450165 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.436107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.5.71.129:40735)
0923 20:46:13.437069 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.437080 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.437110 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:46:13.437207 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.437216 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.437469 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.438442 (+ 973us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:13.438640 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.440698 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.444069 (+ 3371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.446187 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.449123 (+ 2936us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.449439 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.449538 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.449826 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":673,"threads_started":1}
I20260923 20:46:13.453924 6808 tablet_service.cc:1467] Processing CreateTablet for tablet 52a01bdf8a3e4004bbb52d29592e4ab6 (DEFAULT_TABLE table=test-table [id=99c2792865d8419595b5a094db146578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:13.454833 6669 tablet_service.cc:1467] Processing CreateTablet for tablet 52a01bdf8a3e4004bbb52d29592e4ab6 (DEFAULT_TABLE table=test-table [id=99c2792865d8419595b5a094db146578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:13.455539 6808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a01bdf8a3e4004bbb52d29592e4ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:13.456269 6669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a01bdf8a3e4004bbb52d29592e4ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:13.458196 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.436367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46403 (local address 127.0.0.1:55262)
0923 20:46:13.448348 (+ 11981us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.448376 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.448540 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.451034 (+ 2494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.451038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.451067 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:13.451253 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.451265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.453518 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.453524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.455392 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.455412 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.457363 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.457370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.457559 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.457600 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.457680 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.457858 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11836,"thread_start_us":13268,"threads_started":1}
I20260923 20:46:13.459311 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.437301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.5.71.131:46403)
0923 20:46:13.450352 (+ 13051us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.450362 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.450394 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:46:13.450494 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.450502 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.450788 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.450995 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:13.451491 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.453340 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.455621 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.457136 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.458459 (+ 1323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.458707 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.458802 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.458983 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12871,"thread_start_us":12687,"threads_started":1}
I20260923 20:46:13.463536 6947 tablet_service.cc:1467] Processing CreateTablet for tablet 52a01bdf8a3e4004bbb52d29592e4ab6 (DEFAULT_TABLE table=test-table [id=99c2792865d8419595b5a094db146578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:13.466023 6947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a01bdf8a3e4004bbb52d29592e4ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:13.469764 7037 tablet_bootstrap.cc:492] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Bootstrap starting.
I20260923 20:46:13.471701 7036 tablet_bootstrap.cc:492] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Bootstrap starting.
I20260923 20:46:13.473690 7037 tablet_bootstrap.cc:654] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:13.475157 7036 tablet_bootstrap.cc:654] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:13.475158 7037 log.cc:824] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:13.476436 7036 log.cc:824] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:13.478377 7037 tablet_bootstrap.cc:492] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: No bootstrap required, opened a new log
I20260923 20:46:13.478683 7037 ts_tablet_manager.cc:1397] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260923 20:46:13.479715 7036 tablet_bootstrap.cc:492] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: No bootstrap required, opened a new log
I20260923 20:46:13.480031 7036 ts_tablet_manager.cc:1397] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260923 20:46:13.482611 7040 tablet_bootstrap.cc:492] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Bootstrap starting.
I20260923 20:46:13.488933 7040 tablet_bootstrap.cc:654] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:13.490947 7037 raft_consensus.cc:348] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.491528 7040 log.cc:824] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:13.491595 7037 raft_consensus.cc:374] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:13.491817 7037 raft_consensus.cc:729] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67945d63a7a84b4fb01325688202d30a, State: Initialized, Role: FOLLOWER
I20260923 20:46:13.492741 7037 consensus_queue.cc:260] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.493604 7036 raft_consensus.cc:348] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.494617 7036 raft_consensus.cc:374] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:13.494832 7036 raft_consensus.cc:729] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372c8b3b589c477ca014990f930b36bb, State: Initialized, Role: FOLLOWER
I20260923 20:46:13.495844 7036 consensus_queue.cc:260] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.496964 7037 ts_tablet_manager.cc:1428] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Time spent starting tablet: real 0.018s user 0.012s sys 0.006s
I20260923 20:46:13.499198 6878 heartbeater.cc:503] Master 127.5.71.190:38467 was elected leader, sending a full tablet report...
I20260923 20:46:13.501793 7036 ts_tablet_manager.cc:1428] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Time spent starting tablet: real 0.022s user 0.014s sys 0.005s
I20260923 20:46:13.501910 7040 tablet_bootstrap.cc:492] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: No bootstrap required, opened a new log
I20260923 20:46:13.502316 7040 ts_tablet_manager.cc:1397] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.012s
I20260923 20:46:13.513741 7040 raft_consensus.cc:348] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.514285 7040 raft_consensus.cc:374] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:13.514456 7040 raft_consensus.cc:729] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3078ef8c3ef47c2a2df3e1a2d0a00e8, State: Initialized, Role: FOLLOWER
I20260923 20:46:13.515177 7040 consensus_queue.cc:260] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.516881 7017 heartbeater.cc:503] Master 127.5.71.190:38467 was elected leader, sending a full tablet report...
I20260923 20:46:13.517802 7040 ts_tablet_manager.cc:1428] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260923 20:46:13.521135 6740 tablet.cc:2369] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:13.566943 6879 tablet.cc:2369] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:13.598433 7018 tablet.cc:2369] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:13.732079 7044 raft_consensus.cc:482] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:13.732436 7044 raft_consensus.cc:504] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.734304 7044 leader_election.cc:288] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645), 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735)
I20260923 20:46:13.740548 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.734750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:36645 (local address 127.5.71.131:45581)
0923 20:46:13.734983 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.735003 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.735109 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.736033 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.736037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.736063 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:13.736321 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.736334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.737376 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.737381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.738684 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.738696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.739814 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.739821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.740001 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.740024 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.740133 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.740264 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 20:46:13.740548 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.734678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:45581 (local address 127.5.71.130:36645)
0923 20:46:13.734922 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.734939 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.735089 (+ 150us) server_negotiation.cc:407] Connection header received
0923 20:46:13.735323 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.735344 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.735466 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.735817 (+ 351us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:13.736477 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.737231 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.738848 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.739635 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.739829 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.739909 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.740172 (+ 263us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.740295 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:46:13.741708 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.735858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:58257 (local address 127.5.71.129:40735)
0923 20:46:13.736096 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.736102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.736374 (+ 272us) server_negotiation.cc:407] Connection header received
0923 20:46:13.736556 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.736561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.736664 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.736767 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:13.737291 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.738118 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.739737 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.740951 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.741228 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.741308 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.741411 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.741500 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 20:46:13.742043 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.735300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40735 (local address 127.5.71.131:58257)
0923 20:46:13.736228 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:13.736245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:13.736410 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:13.736778 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:13.736782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:13.736805 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:13.737137 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.737145 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.738253 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.738260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.739574 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:13.739585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.741113 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.741117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:13.741234 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.741256 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 20:46:13.741382 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 20:46:13.741546 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":1,"thread_start_us":757,"threads_started":1}
I20260923 20:46:13.742084 6833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a01bdf8a3e4004bbb52d29592e4ab6" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c8b3b589c477ca014990f930b36bb" is_pre_election: true
I20260923 20:46:13.743110 6833 raft_consensus.cc:2454] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 0.
I20260923 20:46:13.743551 6694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a01bdf8a3e4004bbb52d29592e4ab6" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67945d63a7a84b4fb01325688202d30a" is_pre_election: true
I20260923 20:46:13.744007 6900 leader_election.cc:302] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372c8b3b589c477ca014990f930b36bb, b3078ef8c3ef47c2a2df3e1a2d0a00e8; no voters:
I20260923 20:46:13.744496 7044 raft_consensus.cc:2781] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:13.744434 6694 raft_consensus.cc:2454] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 0.
I20260923 20:46:13.744706 7044 raft_consensus.cc:482] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:13.744861 7044 raft_consensus.cc:3037] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:13.748214 7044 raft_consensus.cc:504] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.749037 7044 leader_election.cc:288] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 election: Requested vote from peers 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645), 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735)
I20260923 20:46:13.749719 6694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a01bdf8a3e4004bbb52d29592e4ab6" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67945d63a7a84b4fb01325688202d30a"
I20260923 20:46:13.749773 6833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a01bdf8a3e4004bbb52d29592e4ab6" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c8b3b589c477ca014990f930b36bb"
I20260923 20:46:13.749977 6694 raft_consensus.cc:3037] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:13.750015 6833 raft_consensus.cc:3037] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:13.753216 6833 raft_consensus.cc:2454] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 1.
I20260923 20:46:13.753724 6694 raft_consensus.cc:2454] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 1.
I20260923 20:46:13.753759 6900 leader_election.cc:302] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372c8b3b589c477ca014990f930b36bb, b3078ef8c3ef47c2a2df3e1a2d0a00e8; no voters:
I20260923 20:46:13.754248 7044 raft_consensus.cc:2781] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:13.754922 7044 raft_consensus.cc:686] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 LEADER]: Becoming Leader. State: Replica: b3078ef8c3ef47c2a2df3e1a2d0a00e8, State: Running, Role: LEADER
I20260923 20:46:13.755654 7044 consensus_queue.cc:237] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.761510 6531 catalog_manager.cc:5495] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } health_report { overall_health: HEALTHY } } }
I20260923 20:46:13.834139 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:13.827427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.5.71.131:46403)
0923 20:46:13.827696 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:13.827702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:13.827980 (+ 278us) server_negotiation.cc:407] Connection header received
0923 20:46:13.828325 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:13.828329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:13.828408 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:13.828541 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:13.828957 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.830113 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.831865 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:13.832315 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:13.832414 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:13.832977 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:13.833555 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:13.833668 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:13.833813 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 20:46:13.833878 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:46:13.843587 6833 raft_consensus.cc:1261] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Refusing update from remote peer b3078ef8c3ef47c2a2df3e1a2d0a00e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:13.844038 6694 raft_consensus.cc:1261] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Refusing update from remote peer b3078ef8c3ef47c2a2df3e1a2d0a00e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:13.844688 7044 consensus_queue.cc:1035] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:13.845329 7046 consensus_queue.cc:1035] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:13.880715 6532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:13.880980 6532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:13.885254 6532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: Sending DeleteTablet for 3 replicas of tablet 52a01bdf8a3e4004bbb52d29592e4ab6
I20260923 20:46:13.886675 6669 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a01bdf8a3e4004bbb52d29592e4ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:13 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:51142
I20260923 20:46:13.886806 6808 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a01bdf8a3e4004bbb52d29592e4ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:13 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:59478
I20260923 20:46:13.886847 6947 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a01bdf8a3e4004bbb52d29592e4ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:13 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:55262
I20260923 20:46:13.890300 6532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:13.891317 6532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:13.904384 6668 tablet_service.cc:1467] Processing CreateTablet for tablet 993cd874776c4d3da5bf718f1c34342e (DEFAULT_TABLE table=test-table [id=9a78ce20aec944339b3a50797b2c6b90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:13.905241 6668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993cd874776c4d3da5bf718f1c34342e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:13.905978 7061 tablet_replica.cc:330] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: stopping tablet replica
I20260923 20:46:13.907056 7061 raft_consensus.cc:2229] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:13.908452 7061 raft_consensus.cc:2258] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:13.908384 6946 tablet_service.cc:1467] Processing CreateTablet for tablet 993cd874776c4d3da5bf718f1c34342e (DEFAULT_TABLE table=test-table [id=9a78ce20aec944339b3a50797b2c6b90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:13.909192 6946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993cd874776c4d3da5bf718f1c34342e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:13.911723 6807 tablet_service.cc:1467] Processing CreateTablet for tablet 993cd874776c4d3da5bf718f1c34342e (DEFAULT_TABLE table=test-table [id=9a78ce20aec944339b3a50797b2c6b90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:13.912010 7059 tablet_replica.cc:330] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: stopping tablet replica
I20260923 20:46:13.912235 7060 tablet_replica.cc:330] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: stopping tablet replica
I20260923 20:46:13.912366 6807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993cd874776c4d3da5bf718f1c34342e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:13.912508 7059 raft_consensus.cc:2229] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:13.913168 7059 raft_consensus.cc:2258] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:13.914052 7060 raft_consensus.cc:2229] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:13.915645 7061 ts_tablet_manager.cc:1905] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:13.916250 7060 raft_consensus.cc:2258] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:13.919561 7060 ts_tablet_manager.cc:1905] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:13.923017 7040 tablet_bootstrap.cc:492] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Bootstrap starting.
I20260923 20:46:13.923774 7037 tablet_bootstrap.cc:492] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Bootstrap starting.
I20260923 20:46:13.926486 7040 tablet_bootstrap.cc:654] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:13.926468 7037 tablet_bootstrap.cc:654] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:13.930081 7036 tablet_bootstrap.cc:492] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Bootstrap starting.
I20260923 20:46:13.931541 7059 ts_tablet_manager.cc:1905] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:13.934192 7036 tablet_bootstrap.cc:654] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:13.940537 7061 ts_tablet_manager.cc:1918] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:13.940830 7061 log.cc:1196] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/52a01bdf8a3e4004bbb52d29592e4ab6
I20260923 20:46:13.941871 7061 ts_tablet_manager.cc:1939] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 372c8b3b589c477ca014990f930b36bb: Deleting consensus metadata
I20260923 20:46:13.944553 6518 catalog_manager.cc:4841] TS 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645): tablet 52a01bdf8a3e4004bbb52d29592e4ab6 (table test-table [id=99c2792865d8419595b5a094db146578]) successfully deleted
I20260923 20:46:13.946600 7036 tablet_bootstrap.cc:492] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: No bootstrap required, opened a new log
I20260923 20:46:13.946861 7036 ts_tablet_manager.cc:1397] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.004s
I20260923 20:46:13.948560 7036 raft_consensus.cc:348] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.948884 7036 raft_consensus.cc:374] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:13.949132 7036 raft_consensus.cc:729] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372c8b3b589c477ca014990f930b36bb, State: Initialized, Role: FOLLOWER
I20260923 20:46:13.949610 7036 consensus_queue.cc:260] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.953981 7036 ts_tablet_manager.cc:1428] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260923 20:46:13.956275 7037 tablet_bootstrap.cc:492] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: No bootstrap required, opened a new log
I20260923 20:46:13.956578 7037 ts_tablet_manager.cc:1397] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.019s
I20260923 20:46:13.959508 7037 raft_consensus.cc:348] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.960072 7037 raft_consensus.cc:374] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:13.959677 7060 ts_tablet_manager.cc:1918] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:13.960469 7037 raft_consensus.cc:729] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67945d63a7a84b4fb01325688202d30a, State: Initialized, Role: FOLLOWER
I20260923 20:46:13.960834 7060 log.cc:1196] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/52a01bdf8a3e4004bbb52d29592e4ab6
I20260923 20:46:13.961100 7037 consensus_queue.cc:260] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.961884 7060 ts_tablet_manager.cc:1939] T 52a01bdf8a3e4004bbb52d29592e4ab6 P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Deleting consensus metadata
I20260923 20:46:13.962581 7040 tablet_bootstrap.cc:492] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: No bootstrap required, opened a new log
I20260923 20:46:13.962832 7040 ts_tablet_manager.cc:1397] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.021s
I20260923 20:46:13.963299 7059 ts_tablet_manager.cc:1918] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:13.963575 7059 log.cc:1196] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/52a01bdf8a3e4004bbb52d29592e4ab6
I20260923 20:46:13.964515 7059 ts_tablet_manager.cc:1939] T 52a01bdf8a3e4004bbb52d29592e4ab6 P 67945d63a7a84b4fb01325688202d30a: Deleting consensus metadata
I20260923 20:46:13.964510 6518 catalog_manager.cc:4841] TS b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131:46403): tablet 52a01bdf8a3e4004bbb52d29592e4ab6 (table test-table [id=99c2792865d8419595b5a094db146578]) successfully deleted
I20260923 20:46:13.966938 6518 catalog_manager.cc:4841] TS 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735): tablet 52a01bdf8a3e4004bbb52d29592e4ab6 (table test-table [id=99c2792865d8419595b5a094db146578]) successfully deleted
I20260923 20:46:13.964851 7040 raft_consensus.cc:348] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.967639 7040 raft_consensus.cc:374] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:13.967872 7040 raft_consensus.cc:729] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3078ef8c3ef47c2a2df3e1a2d0a00e8, State: Initialized, Role: FOLLOWER
I20260923 20:46:13.967913 7037 ts_tablet_manager.cc:1428] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260923 20:46:13.968346 7040 consensus_queue.cc:260] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:13.969430 7040 ts_tablet_manager.cc:1428] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260923 20:46:14.056541 7042 raft_consensus.cc:482] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:14.056923 7042 raft_consensus.cc:504] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.059608 7042 leader_election.cc:288] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645), b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131:46403)
I20260923 20:46:14.073151 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.061574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:55085 (local address 127.5.71.131:46403)
0923 20:46:14.061741 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.061747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.063136 (+ 1389us) server_negotiation.cc:407] Connection header received
0923 20:46:14.063466 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.063472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.063592 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.063833 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:14.065488 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.066977 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.070213 (+ 3236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.071569 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.071868 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.071996 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.072395 (+ 399us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.072526 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:46:14.073491 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.061419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46403 (local address 127.5.71.129:55085)
0923 20:46:14.062999 (+ 1580us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:14.063021 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:14.063251 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:14.063906 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:14.063912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:14.063942 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:14.065070 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.065087 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.067120 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.067127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.069995 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.070011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.071976 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.071984 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.072214 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.072246 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:46:14.072367 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 20:46:14.072514 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"spinlock_wait_cycles":3328,"thread_start_us":861,"threads_started":1}
I20260923 20:46:14.074293 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.060034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:51931 (local address 127.5.71.130:36645)
0923 20:46:14.060295 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.060302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.061498 (+ 1196us) server_negotiation.cc:407] Connection header received
0923 20:46:14.062133 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.062142 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.062297 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.062364 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:14.064185 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.065479 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.070345 (+ 4866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.072333 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.072513 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.072588 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.073569 (+ 981us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.073763 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260923 20:46:14.074851 6972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cd874776c4d3da5bf718f1c34342e" candidate_uuid: "67945d63a7a84b4fb01325688202d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" is_pre_election: true
I20260923 20:46:14.075634 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.060017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:36645 (local address 127.5.71.129:51931)
0923 20:46:14.061386 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:14.061407 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:14.061869 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:14.062633 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:14.062637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:14.062694 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:14.063885 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.063901 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.065671 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.065676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.070080 (+ 4404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.070097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.073038 (+ 2941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.073046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.073323 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.073360 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:46:14.074591 (+ 1231us) client_negotiation.cc:240] Negotiation successful
0923 20:46:14.074781 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"spinlock_wait_cycles":149248,"thread_start_us":809,"threads_started":1}
I20260923 20:46:14.076025 6972 raft_consensus.cc:2454] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67945d63a7a84b4fb01325688202d30a in term 0.
I20260923 20:46:14.076924 6833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cd874776c4d3da5bf718f1c34342e" candidate_uuid: "67945d63a7a84b4fb01325688202d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c8b3b589c477ca014990f930b36bb" is_pre_election: true
I20260923 20:46:14.077356 6833 raft_consensus.cc:2454] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67945d63a7a84b4fb01325688202d30a in term 0.
I20260923 20:46:14.077301 6621 leader_election.cc:302] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67945d63a7a84b4fb01325688202d30a, b3078ef8c3ef47c2a2df3e1a2d0a00e8; no voters:
I20260923 20:46:14.077953 7042 raft_consensus.cc:2781] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:14.078330 7042 raft_consensus.cc:482] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:14.078536 7042 raft_consensus.cc:3037] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:14.083930 7042 raft_consensus.cc:504] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.085155 7042 leader_election.cc:288] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [CANDIDATE]: Term 1 election: Requested vote from peers 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645), b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131:46403)
I20260923 20:46:14.086294 6972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cd874776c4d3da5bf718f1c34342e" candidate_uuid: "67945d63a7a84b4fb01325688202d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8"
I20260923 20:46:14.086360 6833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993cd874776c4d3da5bf718f1c34342e" candidate_uuid: "67945d63a7a84b4fb01325688202d30a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c8b3b589c477ca014990f930b36bb"
I20260923 20:46:14.086651 6972 raft_consensus.cc:3037] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:14.086753 6833 raft_consensus.cc:3037] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:14.091838 6972 raft_consensus.cc:2454] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67945d63a7a84b4fb01325688202d30a in term 1.
I20260923 20:46:14.091931 6833 raft_consensus.cc:2454] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67945d63a7a84b4fb01325688202d30a in term 1.
I20260923 20:46:14.092545 6621 leader_election.cc:302] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372c8b3b589c477ca014990f930b36bb, 67945d63a7a84b4fb01325688202d30a; no voters:
I20260923 20:46:14.093288 7042 raft_consensus.cc:2781] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:14.094331 7042 raft_consensus.cc:686] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 1 LEADER]: Becoming Leader. State: Replica: 67945d63a7a84b4fb01325688202d30a, State: Running, Role: LEADER
I20260923 20:46:14.095379 7042 consensus_queue.cc:237] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.100826 6531 catalog_manager.cc:5495] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a reported cstate change: term changed from 0 to 1, leader changed from <none> to 67945d63a7a84b4fb01325688202d30a (127.5.71.129). New cstate: current_term: 1 leader_uuid: "67945d63a7a84b4fb01325688202d30a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } health_report { overall_health: UNKNOWN } } }
Sep 23 20:46:14 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196371, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:14.183877 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.155481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.5.71.190:38467)
0923 20:46:14.157069 (+ 1588us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.157079 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.158233 (+ 1154us) server_negotiation.cc:407] Connection header received
0923 20:46:14.158382 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.158388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.158473 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.158655 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:14.160773 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.162536 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.164543 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.165816 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.166074 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.172045 (+ 5971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:14.172055 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:14.172061 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:14.172116 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:14.179037 (+ 6921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.180531 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.180538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.180543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.180813 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.181431 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.181438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.181447 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.181902 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:14.182107 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.182801 (+ 694us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.183161 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":1359,"thread_start_us":1085,"threads_started":1}
I20260923 20:46:14.183961 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.155514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38467 (local address 127.0.0.1:38304)
0923 20:46:14.158068 (+ 2554us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:14.158102 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:14.158262 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:14.158677 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:14.158680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:14.159837 (+ 1157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:14.160533 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.160545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.162756 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.162761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.164387 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.164401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.166094 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.166101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.166607 (+ 506us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.167917 (+ 1310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:14.167934 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:14.171612 (+ 3678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:14.179293 (+ 7681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:14.179313 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:14.179321 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:14.180352 (+ 1031us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:14.181005 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:14.181009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:14.181012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:14.181267 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:14.182099 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:14.182108 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:14.182340 (+ 232us) client_negotiation.cc:769] Sending connection context
0923 20:46:14.182613 (+ 273us) client_negotiation.cc:240] Negotiation successful
0923 20:46:14.182974 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2376,"thread_start_us":2113,"threads_started":1}
I20260923 20:46:14.192358 6531 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:38304:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 20:46:14.193946 6531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:14.203747 6668 tablet_service.cc:1467] Processing CreateTablet for tablet 3441974ee68d4d52ad1d8c70edfdedae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=22d0a5f5940f47a2ae01181886afef57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:46:14.203830 6807 tablet_service.cc:1467] Processing CreateTablet for tablet 3441974ee68d4d52ad1d8c70edfdedae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=22d0a5f5940f47a2ae01181886afef57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:46:14.204128 6946 tablet_service.cc:1467] Processing CreateTablet for tablet 3441974ee68d4d52ad1d8c70edfdedae (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=22d0a5f5940f47a2ae01181886afef57]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:46:14.204540 6807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3441974ee68d4d52ad1d8c70edfdedae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:14.204541 6668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3441974ee68d4d52ad1d8c70edfdedae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:14.204934 6946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3441974ee68d4d52ad1d8c70edfdedae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:14.216553 7036 tablet_bootstrap.cc:492] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: Bootstrap starting.
I20260923 20:46:14.217554 7037 tablet_bootstrap.cc:492] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: Bootstrap starting.
I20260923 20:46:14.218746 7040 tablet_bootstrap.cc:492] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Bootstrap starting.
I20260923 20:46:14.220387 7036 tablet_bootstrap.cc:654] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:14.221539 7037 tablet_bootstrap.cc:654] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:14.222029 7040 tablet_bootstrap.cc:654] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:14.234450 7040 tablet_bootstrap.cc:492] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: No bootstrap required, opened a new log
I20260923 20:46:14.234766 7040 ts_tablet_manager.cc:1397] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260923 20:46:14.235241 7036 tablet_bootstrap.cc:492] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: No bootstrap required, opened a new log
I20260923 20:46:14.235564 7036 ts_tablet_manager.cc:1397] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.012s
I20260923 20:46:14.236021 7040 raft_consensus.cc:348] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.236415 7040 raft_consensus.cc:374] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:14.236552 7040 raft_consensus.cc:729] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3078ef8c3ef47c2a2df3e1a2d0a00e8, State: Initialized, Role: FOLLOWER
I20260923 20:46:14.237018 7036 raft_consensus.cc:348] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.236948 7040 consensus_queue.cc:260] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.237365 7036 raft_consensus.cc:374] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:14.237534 7036 raft_consensus.cc:729] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 372c8b3b589c477ca014990f930b36bb, State: Initialized, Role: FOLLOWER
I20260923 20:46:14.237972 7036 consensus_queue.cc:260] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.238509 7040 ts_tablet_manager.cc:1428] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260923 20:46:14.238940 7036 ts_tablet_manager.cc:1428] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260923 20:46:14.237618 7054 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.239997 7054 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.238849 7043 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.241539 7043 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.241809 7037 tablet_bootstrap.cc:492] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: No bootstrap required, opened a new log
I20260923 20:46:14.242113 7037 ts_tablet_manager.cc:1397] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.012s
I20260923 20:46:14.245183 7037 raft_consensus.cc:348] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.245616 7037 raft_consensus.cc:374] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:14.245795 7037 raft_consensus.cc:729] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67945d63a7a84b4fb01325688202d30a, State: Initialized, Role: FOLLOWER
I20260923 20:46:14.246152 7037 consensus_queue.cc:260] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.246656 7042 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.246870 7037 ts_tablet_manager.cc:1428] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260923 20:46:14.247066 7042 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.557324 7054 raft_consensus.cc:482] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:14.557626 7054 raft_consensus.cc:504] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.558343 7054 leader_election.cc:288] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735), 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645)
I20260923 20:46:14.559167 6694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3441974ee68d4d52ad1d8c70edfdedae" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67945d63a7a84b4fb01325688202d30a" is_pre_election: true
I20260923 20:46:14.559144 6833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3441974ee68d4d52ad1d8c70edfdedae" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c8b3b589c477ca014990f930b36bb" is_pre_election: true
I20260923 20:46:14.559424 6694 raft_consensus.cc:2454] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 0.
I20260923 20:46:14.559443 6833 raft_consensus.cc:2454] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 0.
I20260923 20:46:14.559960 6900 leader_election.cc:302] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372c8b3b589c477ca014990f930b36bb, b3078ef8c3ef47c2a2df3e1a2d0a00e8; no voters:
I20260923 20:46:14.560233 7054 raft_consensus.cc:2781] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:14.560361 7054 raft_consensus.cc:482] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:14.560458 7054 raft_consensus.cc:3037] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:14.562595 7054 raft_consensus.cc:504] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.563050 7054 leader_election.cc:288] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 election: Requested vote from peers 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735), 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645)
I20260923 20:46:14.563623 6694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3441974ee68d4d52ad1d8c70edfdedae" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67945d63a7a84b4fb01325688202d30a"
I20260923 20:46:14.563709 6833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3441974ee68d4d52ad1d8c70edfdedae" candidate_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "372c8b3b589c477ca014990f930b36bb"
I20260923 20:46:14.563875 6694 raft_consensus.cc:3037] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:14.563951 6833 raft_consensus.cc:3037] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:14.566781 6694 raft_consensus.cc:2454] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 1.
I20260923 20:46:14.566767 6833 raft_consensus.cc:2454] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3078ef8c3ef47c2a2df3e1a2d0a00e8 in term 1.
I20260923 20:46:14.567198 6900 leader_election.cc:302] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 372c8b3b589c477ca014990f930b36bb, b3078ef8c3ef47c2a2df3e1a2d0a00e8; no voters:
I20260923 20:46:14.567467 7054 raft_consensus.cc:2781] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:14.567971 7054 raft_consensus.cc:686] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 LEADER]: Becoming Leader. State: Replica: b3078ef8c3ef47c2a2df3e1a2d0a00e8, State: Running, Role: LEADER
I20260923 20:46:14.568245 7054 consensus_queue.cc:237] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } }
I20260923 20:46:14.571475 7079 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: TxnStatusTablet state changed. Reason: New leader b3078ef8c3ef47c2a2df3e1a2d0a00e8. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.571894 7079 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:46:14.572899 7042 consensus_queue.cc:1035] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:14.573195 6531 catalog_manager.cc:5495] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } health_report { overall_health: HEALTHY } } }
I20260923 20:46:14.574532 7081 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 20:46:14.574810 7081 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 20:46:14.588754 7067 consensus_queue.cc:1035] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [LEADER]: Connected to new peer: Peer: permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:14.608646 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.602118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46403 (local address 127.0.0.1:40534)
0923 20:46:14.602431 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:14.602462 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:14.602693 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:14.603279 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:14.603283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:14.603295 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:14.603463 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.603472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.605056 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.605060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.606108 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.606121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.607174 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.607179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.607374 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.607441 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.607989 (+ 548us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.608010 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 20:46:14.608086 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 20:46:14.608298 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260923 20:46:14.608686 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.602206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.5.71.131:46403)
0923 20:46:14.602837 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.602844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.602868 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:46:14.602948 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.602953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.603058 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.603168 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:14.603640 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.604877 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.606266 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.606998 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.607129 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.607657 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.607864 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.607967 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.608134 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.608215 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":318,"threads_started":1}
I20260923 20:46:14.613982 6694 raft_consensus.cc:1261] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Refusing update from remote peer b3078ef8c3ef47c2a2df3e1a2d0a00e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:14.614033 6833 raft_consensus.cc:1261] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Refusing update from remote peer b3078ef8c3ef47c2a2df3e1a2d0a00e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:14.614832 7079 consensus_queue.cc:1035] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:14.615233 7054 consensus_queue.cc:1035] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:14.625051 7067 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: TxnStatusTablet state changed. Reason: New leader b3078ef8c3ef47c2a2df3e1a2d0a00e8. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.625406 7067 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.631619 7084 mvcc.cc:205] Tried to move back new op lower bound from 7332644350411104256 to 7332644350233423872. Current Snapshot: MvccSnapshot[applied={T|T < 7332644350411104256}]
I20260923 20:46:14.637840 7054 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.638302 7054 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:46:14.640336 7043 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: TxnStatusTablet state changed. Reason: New leader b3078ef8c3ef47c2a2df3e1a2d0a00e8. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.640626 7043 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.642427 7067 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.642683 7067 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P 67945d63a7a84b4fb01325688202d30a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.649039 7054 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.649361 7054 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P b3078ef8c3ef47c2a2df3e1a2d0a00e8: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:46:14.651279 7088 mvcc.cc:205] Tried to move back new op lower bound from 7332644350411104256 to 7332644350233423872. Current Snapshot: MvccSnapshot[applied={T|T < 7332644350411104256}]
I20260923 20:46:14.652971 7043 tablet_replica.cc:439] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67945d63a7a84b4fb01325688202d30a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40735 } } peers { permanent_uuid: "372c8b3b589c477ca014990f930b36bb" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 36645 } } peers { permanent_uuid: "b3078ef8c3ef47c2a2df3e1a2d0a00e8" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46403 } } }
I20260923 20:46:14.653240 7043 tablet_replica.cc:442] T 3441974ee68d4d52ad1d8c70edfdedae P 372c8b3b589c477ca014990f930b36bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:14.672143 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.656071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.5.71.129:40735)
0923 20:46:14.664531 (+ 8460us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.664542 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.665230 (+ 688us) server_negotiation.cc:407] Connection header received
0923 20:46:14.665454 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.665460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.665577 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.665830 (+ 253us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:14.666617 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.667543 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.669614 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.670148 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.670306 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.670748 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.671413 (+ 665us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.671526 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.671671 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.671742 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8213,"thread_start_us":8011,"threads_started":1}
I20260923 20:46:14.689083 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.682680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:54315 (local address 127.5.71.131:46403)
0923 20:46:14.682880 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.682886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.683499 (+ 613us) server_negotiation.cc:407] Connection header received
0923 20:46:14.683669 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.683675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.683771 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.683889 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:14.684307 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.685480 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.686946 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.687648 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.687811 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.688351 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.688480 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.688569 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.688690 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.688801 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 20:46:14.689117 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.682652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46403 (local address 127.5.71.129:54315)
0923 20:46:14.683398 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:14.683416 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:14.683541 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:14.683900 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:14.683903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:14.683913 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:14.684166 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.684173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.685607 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.685611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.686806 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.686819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.687869 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.687875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.688021 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.688082 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.688577 (+ 495us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.688585 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 20:46:14.688644 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 20:46:14.688800 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":366,"threads_started":1}
I20260923 20:46:14.736948 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.730524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:53415 (local address 127.5.71.129:40735)
0923 20:46:14.731096 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.731101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.731117 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:46:14.731173 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.731176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.731238 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.731308 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:14.732015 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.732719 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.735177 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.735690 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.735847 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.736336 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.736435 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.736505 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.736663 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.736725 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447}
I20260923 20:46:14.737215 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.729876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40735 (local address 127.5.71.131:53415)
0923 20:46:14.730621 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:14.730647 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:14.730794 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:14.731371 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:14.731377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:14.731391 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:14.731845 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.731859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.732936 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.732942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.734797 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:14.734812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.735891 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.735898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:14.736068 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.736167 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:14.736541 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:14.736554 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 20:46:14.736653 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 20:46:14.736807 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":698,"threads_started":1}
I20260923 20:46:14.782218 6531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:14.782500 6531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:14.786238 6531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b9120c605e344a2a0a6135d93c0bd02: Sending DeleteTablet for 3 replicas of tablet 993cd874776c4d3da5bf718f1c34342e
I20260923 20:46:14.787334 6807 tablet_service.cc:1514] Processing DeleteTablet for tablet 993cd874776c4d3da5bf718f1c34342e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:14 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:59478
I20260923 20:46:14.788376 6946 tablet_service.cc:1514] Processing DeleteTablet for tablet 993cd874776c4d3da5bf718f1c34342e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:14 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:55262
I20260923 20:46:14.789146 6668 tablet_service.cc:1514] Processing DeleteTablet for tablet 993cd874776c4d3da5bf718f1c34342e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:14 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:51142
I20260923 20:46:14.807641 7107 tablet_replica.cc:330] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: stopping tablet replica
I20260923 20:46:14.807991 7110 tablet_replica.cc:330] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: stopping tablet replica
I20260923 20:46:14.808314 7110 raft_consensus.cc:2229] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:14.808800 7107 raft_consensus.cc:2229] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:14.809330 7107 raft_consensus.cc:2258] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:14.809888 7108 tablet_replica.cc:330] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: stopping tablet replica
I20260923 20:46:14.810169 7108 raft_consensus.cc:2229] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:14.810422 7110 raft_consensus.cc:2258] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:14.810932 7108 raft_consensus.cc:2258] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:14.811146 7107 ts_tablet_manager.cc:1905] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:14.812016 7110 ts_tablet_manager.cc:1905] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:14.821142 7108 ts_tablet_manager.cc:1905] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:14.821225 7107 ts_tablet_manager.cc:1918] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:14.828059 7107 log.cc:1196] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/993cd874776c4d3da5bf718f1c34342e
I20260923 20:46:14.829488 7107 ts_tablet_manager.cc:1939] T 993cd874776c4d3da5bf718f1c34342e P 372c8b3b589c477ca014990f930b36bb: Deleting consensus metadata
I20260923 20:46:14.831009 6518 catalog_manager.cc:4841] TS 372c8b3b589c477ca014990f930b36bb (127.5.71.130:36645): tablet 993cd874776c4d3da5bf718f1c34342e (table test-table [id=9a78ce20aec944339b3a50797b2c6b90]) successfully deleted
Sep 23 20:46:14 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:14.843581 7108 ts_tablet_manager.cc:1918] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:14.843755 7108 log.cc:1196] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/993cd874776c4d3da5bf718f1c34342e
I20260923 20:46:14.844411 7108 ts_tablet_manager.cc:1939] T 993cd874776c4d3da5bf718f1c34342e P b3078ef8c3ef47c2a2df3e1a2d0a00e8: Deleting consensus metadata
I20260923 20:46:14.846186 6518 catalog_manager.cc:4841] TS b3078ef8c3ef47c2a2df3e1a2d0a00e8 (127.5.71.131:46403): tablet 993cd874776c4d3da5bf718f1c34342e (table test-table [id=9a78ce20aec944339b3a50797b2c6b90]) successfully deleted
I20260923 20:46:14.858538 7110 ts_tablet_manager.cc:1918] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:14.860471 7110 log.cc:1196] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/993cd874776c4d3da5bf718f1c34342e
I20260923 20:46:14.861567 7110 ts_tablet_manager.cc:1939] T 993cd874776c4d3da5bf718f1c34342e P 67945d63a7a84b4fb01325688202d30a: Deleting consensus metadata
I20260923 20:46:14.864181 6518 catalog_manager.cc:4841] TS 67945d63a7a84b4fb01325688202d30a (127.5.71.129:40735): tablet 993cd874776c4d3da5bf718f1c34342e (table test-table [id=9a78ce20aec944339b3a50797b2c6b90]) successfully deleted
I20260923 20:46:14.868757 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.808728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60644 (local address 127.5.71.129:40735)
0923 20:46:14.809112 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.809118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.822569 (+ 13451us) server_negotiation.cc:407] Connection header received
0923 20:46:14.823041 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.823048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.823131 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.823260 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:14.831631 (+ 8371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.833071 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.835105 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.836140 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.836266 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.842471 (+ 6205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:14.842492 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:14.842520 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:14.842572 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:14.855578 (+ 13006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.862113 (+ 6535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.862127 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.862132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.862336 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.864601 (+ 2265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.864609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.864613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.865512 (+ 899us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:14.866021 (+ 509us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.867990 (+ 1969us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.868281 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260923 20:46:14.869818 6734 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60644
I20260923 20:46:14.900820 7114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.873934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.5.71.190:38467)
0923 20:46:14.878542 (+ 4608us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.878549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.878572 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:46:14.878705 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.878712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.878825 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.878964 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:14.882730 (+ 3766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.884863 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.887621 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.889090 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.889372 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.894036 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:14.894043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:14.894048 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:14.894108 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:14.897616 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.898523 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.898529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.898533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.898714 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.899136 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.899142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.899145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.899510 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:14.899822 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.900188 (+ 366us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.900428 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4402,"thread_start_us":4193,"threads_started":1}
W20260923 20:46:14.901469 6531 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51732
I20260923 20:46:14.904464 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6603
I20260923 20:46:14.924273 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.905663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.5.71.129:40735)
0923 20:46:14.905993 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.905999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.908266 (+ 2267us) server_negotiation.cc:407] Connection header received
0923 20:46:14.908456 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.908462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.908564 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.908688 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:14.910256 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.911389 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.913281 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.914000 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.914143 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.917227 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:14.917292 (+ 65us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:14.917297 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:14.917346 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:14.921011 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.921914 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.921920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.921924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.922100 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.922559 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.922564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.922568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.922934 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:14.923256 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.923548 (+ 292us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.923863 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
W20260923 20:46:14.925036 6734 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60646
W20260923 20:46:14.925673 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6603: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:14.925869 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6603
I20260923 20:46:14.945647 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6743
Sep 23 20:46:14 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:14.962080 7116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.946537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49420 (local address 127.5.71.130:36645)
0923 20:46:14.947469 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.947473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.947489 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:46:14.947540 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.947544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.947602 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.947747 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:14.949023 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.949797 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.951053 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.951497 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.951636 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.955944 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:14.955961 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:14.955979 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:14.956019 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:14.958982 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.959742 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.959750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.959752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.959952 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:14.960357 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:14.960364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:14.960367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:14.960745 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:14.961139 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:14.961436 (+ 297us) server_negotiation.cc:299] Negotiation successful
0923 20:46:14.961675 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":659,"threads_started":1}
W20260923 20:46:14.962826 6873 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49420
W20260923 20:46:14.963550 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6743: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:14.963757 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6743
I20260923 20:46:14.982690 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6882
Sep 23 20:46:14 dist-test-slave-dn29 krb5kdc[6487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:15.002717 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:14.983584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.5.71.131:46403)
0923 20:46:14.983841 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:14.983852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:14.983877 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:46:14.984048 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:14.984055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:14.984175 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:14.984313 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:14.985689 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.986959 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.988571 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:14.989257 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:14.989409 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:14.994243 (+ 4834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:14.994266 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:14.994297 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:14.994353 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:14.998989 (+ 4636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:15.000120 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:15.000131 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:15.000135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:15.000372 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:15.000864 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:15.000870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:15.000873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:15.001371 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:15.001722 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:15.002042 (+ 320us) server_negotiation.cc:299] Negotiation successful
0923 20:46:15.002301 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260923 20:46:15.003602 7012 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40546
W20260923 20:46:15.004474 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6882: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:15.004664 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6882
I20260923 20:46:15.028687 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 6500
I20260923 20:46:15.030124 6572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:15.155663 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 6500
2026-09-23T20:46:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5290 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:15 dist-test-slave-dn29 krb5kdc[7120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:15 dist-test-slave-dn29 krb5kdc[7120](info): set up 2 sockets
Sep 23 20:46:15 dist-test-slave-dn29 krb5kdc[7120](info): commencing operation
krb5kdc: starting...
W20260923 20:46:16.248831 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:16 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:16.453003 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:44639
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:44639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:16.787135 7133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:16.787532 7133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:16.787708 7133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:16.795228 7133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:16.795389 7133 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 20:46:16.795493 7133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:16.795617 7133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:16.795729 7133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:16.795851 7133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:16.805914 7133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:44639
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:44639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:16.807313 7133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:16.808790 7133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:16.821094 7138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:16.821075 7139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:16.821144 7141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:16.822209 7133 server_base.cc:1034] running on GCE node
I20260923 20:46:16.823086 7133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:16.824705 7133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:16.826027 7133 hybrid_clock.cc:648] HybridClock initialized: now 1790196376825965 us; error 41 us; skew 500 ppm
I20260923 20:46:16.826438 7133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:16 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196376, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:16.836723 7133 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:16.838452 7133 webserver.cc:460] Webserver started at http://127.5.71.190:44745/ using document root <none> and password file <none>
I20260923 20:46:16.839205 7133 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:16.839293 7133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:16.839550 7133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:16.842911 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "9a33c53d75274df29c88584a6d1e421c"
format_stamp: "Formatted at 2026-09-23 20:46:16 on dist-test-slave-dn29"
server_key: "86a5ae3a0e660028e279acb0735c2cb7"
server_key_iv: "b275c56dded375d2d8d50c240a899aec"
server_key_version: "encryptionkey@0"
I20260923 20:46:16.843659 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "9a33c53d75274df29c88584a6d1e421c"
format_stamp: "Formatted at 2026-09-23 20:46:16 on dist-test-slave-dn29"
server_key: "86a5ae3a0e660028e279acb0735c2cb7"
server_key_iv: "b275c56dded375d2d8d50c240a899aec"
server_key_version: "encryptionkey@0"
I20260923 20:46:16.849416 7133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260923 20:46:16.853602 7148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:16.855168 7133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:46:16.855368 7133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "9a33c53d75274df29c88584a6d1e421c"
format_stamp: "Formatted at 2026-09-23 20:46:16 on dist-test-slave-dn29"
server_key: "86a5ae3a0e660028e279acb0735c2cb7"
server_key_iv: "b275c56dded375d2d8d50c240a899aec"
server_key_version: "encryptionkey@0"
I20260923 20:46:16.855619 7133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:16.868629 7133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:16.873395 7133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:16.873692 7133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:16.897037 7133 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:44639
I20260923 20:46:16.897063 7209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:44639 every 8 connection(s)
I20260923 20:46:16.898466 7133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:16.902375 7210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:16.903177 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 7133
I20260923 20:46:16.903388 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:16.904076 5406 external_mini_cluster.cc:1428] Setting key ac8f8410244c2a02c853869a5976069d
I20260923 20:46:16.914911 7210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Bootstrap starting.
Sep 23 20:46:16 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:16.918075 7210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:16.919121 7210 log.cc:824] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:16.921991 7210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: No bootstrap required, opened a new log
I20260923 20:46:16.925992 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:16.906096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.5.71.190:44639)
0923 20:46:16.906883 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:16.906899 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:16.906944 (+ 45us) server_negotiation.cc:407] Connection header received
0923 20:46:16.908798 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:16.908825 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:16.909247 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:16.909674 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:16.911114 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:16.912569 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:16.913803 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:16.914484 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:16.914685 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:16.918708 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:16.918716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:16.918732 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:16.918775 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:16.921834 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:16.922574 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:16.922582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:16.922588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:16.922750 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:16.923146 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:16.923155 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:16.923157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:16.923695 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:16.924021 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:16.924530 (+ 509us) server_negotiation.cc:299] Negotiation successful
0923 20:46:16.924957 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":332,"threads_started":1}
I20260923 20:46:16.930519 7210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a33c53d75274df29c88584a6d1e421c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44639 } }
I20260923 20:46:16.930898 7210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:16.931041 7210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a33c53d75274df29c88584a6d1e421c, State: Initialized, Role: FOLLOWER
I20260923 20:46:16.931607 7210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a33c53d75274df29c88584a6d1e421c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44639 } }
I20260923 20:46:16.931865 7210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:16.932015 7210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:16.932189 7210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:16.934314 7210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a33c53d75274df29c88584a6d1e421c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44639 } }
I20260923 20:46:16.934819 7210 leader_election.cc:302] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a33c53d75274df29c88584a6d1e421c; no voters:
I20260923 20:46:16.935487 7210 leader_election.cc:288] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:16.935614 7215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:16.936343 7215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [term 1 LEADER]: Becoming Leader. State: Replica: 9a33c53d75274df29c88584a6d1e421c, State: Running, Role: LEADER
I20260923 20:46:16.936964 7215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a33c53d75274df29c88584a6d1e421c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44639 } }
I20260923 20:46:16.937924 7210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:16.940507 7216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a33c53d75274df29c88584a6d1e421c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a33c53d75274df29c88584a6d1e421c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44639 } } }
I20260923 20:46:16.940949 7216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:16.940605 7217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a33c53d75274df29c88584a6d1e421c. Latest consensus state: current_term: 1 leader_uuid: "9a33c53d75274df29c88584a6d1e421c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a33c53d75274df29c88584a6d1e421c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44639 } } }
I20260923 20:46:16.942118 7217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:16.943426 7221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:16.949062 7221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:16.959306 7221 catalog_manager.cc:1347] Generated new cluster ID: f6219a9e4e8842ffadff27a69bbad7c1
I20260923 20:46:16.959458 7221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:16.982335 7221 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:16.984381 7221 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:16.995054 7221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Generated new TSK 0
I20260923 20:46:16.996253 7221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:17.048460 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44639
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:17.394935 7236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:17.395246 7236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:17.395428 7236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:17.403044 7236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:17.403169 7236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:17.403318 7236 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 20:46:17.403398 7236 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:46:17.403502 7236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:17.416064 7236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:44639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:17.417567 7236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:17.419082 7236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:17.433122 7241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:17.433053 7244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:17.433166 7242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:17.433995 7236 server_base.cc:1034] running on GCE node
I20260923 20:46:17.434680 7236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:17.435720 7236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:17.437032 7236 hybrid_clock.cc:648] HybridClock initialized: now 1790196377436947 us; error 70 us; skew 500 ppm
I20260923 20:46:17.437458 7236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:17 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196377, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:17.441370 7236 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:17.443091 7236 webserver.cc:460] Webserver started at http://127.5.71.129:37201/ using document root <none> and password file <none>
I20260923 20:46:17.443928 7236 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:17.444044 7236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:17.444382 7236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:17.447865 7236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
format_stamp: "Formatted at 2026-09-23 20:46:17 on dist-test-slave-dn29"
server_key: "ba9b47c58815f821b3b85730bcd8c1b5"
server_key_iv: "1e1ba0a2bb549010baea174758ea5538"
server_key_version: "encryptionkey@0"
I20260923 20:46:17.448771 7236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
format_stamp: "Formatted at 2026-09-23 20:46:17 on dist-test-slave-dn29"
server_key: "ba9b47c58815f821b3b85730bcd8c1b5"
server_key_iv: "1e1ba0a2bb549010baea174758ea5538"
server_key_version: "encryptionkey@0"
I20260923 20:46:17.454865 7236 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260923 20:46:17.459115 7251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:17.460991 7236 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260923 20:46:17.461184 7236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
format_stamp: "Formatted at 2026-09-23 20:46:17 on dist-test-slave-dn29"
server_key: "ba9b47c58815f821b3b85730bcd8c1b5"
server_key_iv: "1e1ba0a2bb549010baea174758ea5538"
server_key_version: "encryptionkey@0"
I20260923 20:46:17.461392 7236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:17.477310 7236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:17.482125 7236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:17.482465 7236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:17.485553 7236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:17.485692 7236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:17.485852 7236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:17.485960 7236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:46:17 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196377, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:17.515368 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:17.486387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40475 (local address 127.5.71.190:44639)
0923 20:46:17.486870 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:17.486877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:17.488352 (+ 1475us) server_negotiation.cc:407] Connection header received
0923 20:46:17.491685 (+ 3333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:17.491695 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:17.491833 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:17.491963 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:17.494588 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.495622 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.498428 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.498845 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.498997 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:17.504999 (+ 6002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:17.505009 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:17.505018 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:17.505070 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:17.509539 (+ 4469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:17.510779 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:17.510786 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:17.510789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:17.510963 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:17.511668 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:17.511673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:17.511677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:17.512047 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:17.512213 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:17.514803 (+ 2590us) server_negotiation.cc:299] Negotiation successful
0923 20:46:17.515040 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260923 20:46:17.515661 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:17.487067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.5.71.129:40475)
0923 20:46:17.488119 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:17.488187 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:17.491209 (+ 3022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:17.492412 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:17.492427 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:17.493355 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:17.494346 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:17.494376 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.495826 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.495832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:17.498229 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:17.498245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.499121 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.499129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:17.499512 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:17.500523 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:17.500551 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:17.504590 (+ 4039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:17.509737 (+ 5147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:17.509748 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:17.509778 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:17.510575 (+ 797us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:17.511143 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:17.511149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:17.511153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:17.511522 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:17.512191 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:17.512200 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:17.512818 (+ 618us) client_negotiation.cc:769] Sending connection context
0923 20:46:17.513176 (+ 358us) client_negotiation.cc:240] Negotiation successful
0923 20:46:17.513891 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":368,"threads_started":1}
I20260923 20:46:17.536657 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:17.521665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:49349 (local address 127.5.71.190:44639)
0923 20:46:17.521943 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:17.521948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:17.522699 (+ 751us) server_negotiation.cc:407] Connection header received
0923 20:46:17.522990 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:17.522995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:17.523066 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:17.523175 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:17.524185 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.525228 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.527113 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.527578 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.527717 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:17.529609 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:17.529615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:17.529618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:17.529649 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:17.533094 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:17.534325 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:17.534333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:17.534336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:17.534488 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:17.535015 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:17.535020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:17.535023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:17.535410 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:17.535580 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:17.536201 (+ 621us) server_negotiation.cc:299] Negotiation successful
0923 20:46:17.536401 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:46:17.536970 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:17.521967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.5.71.129:49349)
0923 20:46:17.522585 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:17.522601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:17.522849 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:17.523192 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:17.523196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:17.523641 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:17.524034 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:17.524045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.525386 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.525390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:17.526963 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:17.526976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:17.527835 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:17.527842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:17.527991 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:17.528640 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:17.528658 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:17.529389 (+ 731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:17.533308 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:17.533316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:17.533324 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:17.534164 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:17.534623 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:17.534631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:17.534635 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:17.534848 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:17.535560 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:17.535572 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:17.535878 (+ 306us) client_negotiation.cc:769] Sending connection context
0923 20:46:17.536089 (+ 211us) client_negotiation.cc:240] Negotiation successful
0923 20:46:17.536432 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":364,"threads_started":1}
I20260923 20:46:17.544329 7236 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:39049
I20260923 20:46:17.544369 7371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:39049 every 8 connection(s)
I20260923 20:46:17.545707 7236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:17.548008 7372 heartbeater.cc:347] Connected to a master server at 127.5.71.190:44639
I20260923 20:46:17.548270 7372 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:17.548933 7372 heartbeater.cc:511] Master 127.5.71.190:44639 requested a full tablet report, sending...
I20260923 20:46:17.550823 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 7236
I20260923 20:46:17.551034 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:17.551519 7165 ts_manager.cc:194] Registered new tserver with Master: 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049)
I20260923 20:46:17.551882 5406 external_mini_cluster.cc:1428] Setting key 90b16defa23fd20b99927d1a96f2eb9f
I20260923 20:46:17.553498 7165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:40475
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:17.600128 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44639
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:17.941988 7376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:17.942294 7376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:17.942453 7376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:17.950111 7376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:17.950243 7376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:17.950385 7376 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 20:46:17.950448 7376 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:46:17.950551 7376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:17.961133 7376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:44639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:17.962482 7376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:17.963896 7376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:17.978026 7382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:17.978250 7381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:17.978452 7384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:17.979615 7376 server_base.cc:1034] running on GCE node
I20260923 20:46:17.980546 7376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:17.981618 7376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:17.982937 7376 hybrid_clock.cc:648] HybridClock initialized: now 1790196377982845 us; error 79 us; skew 500 ppm
I20260923 20:46:17.983321 7376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:17 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196377, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:17.986975 7376 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:17.988714 7376 webserver.cc:460] Webserver started at http://127.5.71.130:38411/ using document root <none> and password file <none>
I20260923 20:46:17.989475 7376 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:17.989629 7376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:17.989965 7376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:17.993106 7376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "4d59002757c647cc95560b1c8fe5710b"
format_stamp: "Formatted at 2026-09-23 20:46:17 on dist-test-slave-dn29"
server_key: "9bee2f42e51d26296a01c56fa98f68f5"
server_key_iv: "67787d2602042b169500a9abcc6d5daa"
server_key_version: "encryptionkey@0"
I20260923 20:46:17.993947 7376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "4d59002757c647cc95560b1c8fe5710b"
format_stamp: "Formatted at 2026-09-23 20:46:17 on dist-test-slave-dn29"
server_key: "9bee2f42e51d26296a01c56fa98f68f5"
server_key_iv: "67787d2602042b169500a9abcc6d5daa"
server_key_version: "encryptionkey@0"
I20260923 20:46:18.000120 7376 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:46:18.004187 7391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:18.005867 7376 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:46:18.006076 7376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "4d59002757c647cc95560b1c8fe5710b"
format_stamp: "Formatted at 2026-09-23 20:46:17 on dist-test-slave-dn29"
server_key: "9bee2f42e51d26296a01c56fa98f68f5"
server_key_iv: "67787d2602042b169500a9abcc6d5daa"
server_key_version: "encryptionkey@0"
I20260923 20:46:18.006353 7376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:18.020035 7376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:18.024384 7376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:18.024690 7376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:18.027356 7376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:18.027511 7376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:18.027657 7376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:18.027851 7376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:46:18 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196377, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:18.057211 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.028379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:60427 (local address 127.5.71.190:44639)
0923 20:46:18.028585 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.028591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.030454 (+ 1863us) server_negotiation.cc:407] Connection header received
0923 20:46:18.033664 (+ 3210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.033674 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.033838 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.033975 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:18.037172 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.038068 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.041877 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.042697 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.042876 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.049031 (+ 6155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:18.049040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:18.049045 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:18.049099 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:18.052224 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.053716 (+ 1492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.053726 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.053732 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.053920 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.054787 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.054803 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.054807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.055189 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:18.055389 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.056503 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.056769 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:46:18.058429 7400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.028969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.5.71.130:60427)
0923 20:46:18.030222 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.030290 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.033201 (+ 2911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.034558 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.034573 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.035551 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:18.036901 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.036942 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.038312 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.038321 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.041594 (+ 3273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.041626 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.043328 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.043338 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.043697 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.044697 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:18.044743 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:18.048664 (+ 3921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:18.052473 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.052491 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.052520 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.053473 (+ 953us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.054069 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.054075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.054079 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.054607 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.055366 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:18.055376 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:18.056049 (+ 673us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.056403 (+ 354us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.057083 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":627,"threads_started":1}
I20260923 20:46:18.083413 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.065704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:51839 (local address 127.5.71.190:44639)
0923 20:46:18.065995 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.066005 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.066651 (+ 646us) server_negotiation.cc:407] Connection header received
0923 20:46:18.066810 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.066819 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.066934 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.067105 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:18.068323 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.069846 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.072216 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.072893 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.073014 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.076034 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:18.076039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:18.076043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:18.076114 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:18.079666 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.080691 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.080697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.080700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.080893 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.081421 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.081426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.081430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.082102 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:18.082286 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.082750 (+ 464us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.082997 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260923 20:46:18.083520 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.065749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.5.71.130:51839)
0923 20:46:18.066520 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.066555 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.066692 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.067145 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.067148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.067716 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:18.068136 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.068147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.070148 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.070155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.071959 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.072000 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.073344 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.073351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.073578 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.074439 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:18.074469 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:18.075764 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:18.079847 (+ 4083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.079853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.079857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.080531 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.081073 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.081081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.081087 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.081270 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.082239 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:18.082244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:18.082464 (+ 220us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.082625 (+ 161us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.083046 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":426,"threads_started":1}
I20260923 20:46:18.096710 7376 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38519
I20260923 20:46:18.096712 7510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38519 every 8 connection(s)
I20260923 20:46:18.098201 7376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:18.100553 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 7376
I20260923 20:46:18.100751 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:18.100975 7511 heartbeater.cc:347] Connected to a master server at 127.5.71.190:44639
I20260923 20:46:18.101223 7511 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:18.101343 5406 external_mini_cluster.cc:1428] Setting key b1c40568cf370c03402bef4583a542df
I20260923 20:46:18.102159 7511 heartbeater.cc:511] Master 127.5.71.190:44639 requested a full tablet report, sending...
I20260923 20:46:18.103747 7164 ts_manager.cc:194] Registered new tserver with Master: 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:18.105091 7164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:60427
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:18.147117 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44639
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:18.487207 7515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:18.487519 7515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:18.487620 7515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:18.495059 7515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:18.495186 7515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:18.495273 7515 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 20:46:18.495317 7515 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:46:18.495388 7515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:18.505690 7515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:44639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:18.506978 7515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:18.508384 7515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:18.524691 7520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:18.524842 7523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:18.524948 7521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:18.526019 7515 server_base.cc:1034] running on GCE node
I20260923 20:46:18.526794 7515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:18.527932 7515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:18.529182 7515 hybrid_clock.cc:648] HybridClock initialized: now 1790196378529119 us; error 53 us; skew 500 ppm
I20260923 20:46:18.529547 7515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:18 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:18.532944 7515 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:18.534575 7515 webserver.cc:460] Webserver started at http://127.5.71.131:33765/ using document root <none> and password file <none>
I20260923 20:46:18.535343 7515 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:18.535504 7515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:18.535928 7515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:18.539289 7515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "329d0db9e44c4661989fed81051123fa"
format_stamp: "Formatted at 2026-09-23 20:46:18 on dist-test-slave-dn29"
server_key: "5314095de8e3948a95d1649b1fd48b5b"
server_key_iv: "3e1b955ba2b91161d5ad883591ee0320"
server_key_version: "encryptionkey@0"
I20260923 20:46:18.540269 7515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "329d0db9e44c4661989fed81051123fa"
format_stamp: "Formatted at 2026-09-23 20:46:18 on dist-test-slave-dn29"
server_key: "5314095de8e3948a95d1649b1fd48b5b"
server_key_iv: "3e1b955ba2b91161d5ad883591ee0320"
server_key_version: "encryptionkey@0"
I20260923 20:46:18.547771 7515 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260923 20:46:18.551888 7530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:18.553480 7515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:18.553783 7515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "329d0db9e44c4661989fed81051123fa"
format_stamp: "Formatted at 2026-09-23 20:46:18 on dist-test-slave-dn29"
server_key: "5314095de8e3948a95d1649b1fd48b5b"
server_key_iv: "3e1b955ba2b91161d5ad883591ee0320"
server_key_version: "encryptionkey@0"
I20260923 20:46:18.554005 7515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:18.557027 7372 heartbeater.cc:503] Master 127.5.71.190:44639 was elected leader, sending a full tablet report...
I20260923 20:46:18.586311 7515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:18.590952 7515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:18.591266 7515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:18.594174 7515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:18.594312 7515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:18.594453 7515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:18.594553 7515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:46:18 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:18.621658 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.595049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:39433 (local address 127.5.71.190:44639)
0923 20:46:18.595463 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.595470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.596925 (+ 1455us) server_negotiation.cc:407] Connection header received
0923 20:46:18.600235 (+ 3310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.600245 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.600351 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.600475 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:18.603260 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.604378 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.607596 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.608190 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.608391 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.613717 (+ 5326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:18.613723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:18.613727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:18.613766 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:18.617472 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.618445 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.618455 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.618458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.618593 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.619091 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.619097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.619100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.619418 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:18.619615 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.620748 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.621063 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":197,"threads_started":1}
I20260923 20:46:18.622963 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.595701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.5.71.131:39433)
0923 20:46:18.596720 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.596772 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.599775 (+ 3003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.600824 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.600848 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.601865 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:18.603018 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.603057 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.604544 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.604552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.607398 (+ 2846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.607420 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.608556 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.608567 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.608953 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.609895 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:18.609937 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:18.613437 (+ 3500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:18.617652 (+ 4215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.617664 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.617685 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.618274 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.618737 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.618743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.618747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.618980 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.619585 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:18.619595 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:18.620226 (+ 631us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.620634 (+ 408us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.621289 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":347,"threads_started":1}
I20260923 20:46:18.644793 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.628908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:59441 (local address 127.5.71.190:44639)
0923 20:46:18.629147 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.629156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.629891 (+ 735us) server_negotiation.cc:407] Connection header received
0923 20:46:18.630090 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.630093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.630154 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.630269 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:18.631704 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.632640 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.634714 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.635343 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.635477 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.638200 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:18.638206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:18.638211 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:18.638248 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:18.641442 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.642406 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.642413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.642418 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.642589 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.643113 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.643118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.643122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.643425 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:18.643521 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.644142 (+ 621us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.644372 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 20:46:18.644972 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.628975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.5.71.131:59441)
0923 20:46:18.629760 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.629787 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.629962 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.630327 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.630335 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.631055 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:18.631535 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.631551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.632876 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.632880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.634507 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.634527 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.635535 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.635541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.635743 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.636617 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:18.636652 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:18.638005 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:18.641572 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.641578 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.641582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.642212 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.642744 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:18.642747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:18.642752 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:18.642962 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:18.643595 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:18.643598 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:18.643874 (+ 276us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.644035 (+ 161us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.644442 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":532,"threads_started":1}
I20260923 20:46:18.655339 7515 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:41379
I20260923 20:46:18.655378 7650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:41379 every 8 connection(s)
I20260923 20:46:18.656666 7515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:18.658479 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 7515
I20260923 20:46:18.658677 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:18.658963 7651 heartbeater.cc:347] Connected to a master server at 127.5.71.190:44639
I20260923 20:46:18.659204 5406 external_mini_cluster.cc:1428] Setting key 793e2377c2c9bea0bffb4eb135fea171
I20260923 20:46:18.659206 7651 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:18.660048 7651 heartbeater.cc:511] Master 127.5.71.190:44639 requested a full tablet report, sending...
I20260923 20:46:18.661580 7165 ts_manager.cc:194] Registered new tserver with Master: 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379)
I20260923 20:46:18.662664 7165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:39433
I20260923 20:46:18.675169 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 20:46:18 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 20:46:18 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:18.724735 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.709149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.5.71.190:44639)
0923 20:46:18.709400 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.709407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.709521 (+ 114us) server_negotiation.cc:407] Connection header received
0923 20:46:18.709755 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.709761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.709842 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.709960 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:18.711746 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.712539 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.713809 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.714240 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.714384 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.718907 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:18.718914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:18.718919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:18.718973 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:18.721751 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.722512 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.722522 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.722525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.722661 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:18.723088 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:18.723095 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:18.723099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:18.723402 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:18.723715 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.724092 (+ 377us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.724354 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260923 20:46:18.729368 7165 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:18.736454 7165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:18.761260 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.750479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.0.0.1:34318)
0923 20:46:18.751481 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.751498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.751608 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.753061 (+ 1453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.753070 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.753109 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:18.753443 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.753452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.755313 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.755320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.757891 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.757905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.760100 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.760107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.760397 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.760481 (+ 84us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.760645 (+ 164us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.760864 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":763,"threads_started":1}
I20260923 20:46:18.762246 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.749783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.0.0.1:39784)
0923 20:46:18.750649 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.750694 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.750896 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.754766 (+ 3870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.754771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.754809 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:18.755235 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.755254 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.757170 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.757179 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.759341 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.759356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.761392 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.761399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.761584 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.761631 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.761732 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.761891 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":421,"threads_started":1}
I20260923 20:46:18.762246 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.751052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.5.71.130:38519)
0923 20:46:18.752133 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.752139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.752162 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:46:18.752237 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.752242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.752538 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.752811 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:18.753610 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.755104 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.758121 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.759482 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.759662 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.759933 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.761746 (+ 1813us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.761937 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":807,"threads_started":1}
I20260923 20:46:18.762765 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.750217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39784 (local address 127.5.71.131:41379)
0923 20:46:18.754147 (+ 3930us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.754156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.754187 (+ 31us) server_negotiation.cc:407] Connection header received
0923 20:46:18.754274 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.754281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.754534 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.754749 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:18.755494 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.757023 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.759562 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.761172 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.761331 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.761590 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.762071 (+ 481us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.762238 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3765,"thread_start_us":3569,"threads_started":1}
I20260923 20:46:18.764063 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.752101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.5.71.129:39049)
0923 20:46:18.752637 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:18.752648 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:18.754607 (+ 1959us) server_negotiation.cc:407] Connection header received
0923 20:46:18.754881 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:18.754891 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:18.755147 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:18.755370 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:18.756415 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.757895 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.760674 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.762223 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.762375 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.762570 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:18.763553 (+ 983us) server_negotiation.cc:299] Negotiation successful
0923 20:46:18.763737 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":651,"threads_started":1}
I20260923 20:46:18.764115 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:18.751456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.0.0.1:57068)
0923 20:46:18.754474 (+ 3018us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:18.754490 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:18.754650 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:18.755604 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:18.755612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:18.755642 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:18.756176 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.756190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.758055 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.758062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.760499 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:18.760539 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:18.762712 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:18.762718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:18.762904 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:18.762940 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 20:46:18.763021 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 20:46:18.763145 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2902,"thread_start_us":2746,"threads_started":1}
I20260923 20:46:18.766469 7441 tablet_service.cc:1467] Processing CreateTablet for tablet 458d8759c261420aac5c92f0a5817a82 (DEFAULT_TABLE table=test-table [id=1d5108af036c48a899022a88ddb60f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:18.766799 7581 tablet_service.cc:1467] Processing CreateTablet for tablet 458d8759c261420aac5c92f0a5817a82 (DEFAULT_TABLE table=test-table [id=1d5108af036c48a899022a88ddb60f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:18.768126 7581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458d8759c261420aac5c92f0a5817a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:18.768292 7441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458d8759c261420aac5c92f0a5817a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:18.767851 7302 tablet_service.cc:1467] Processing CreateTablet for tablet 458d8759c261420aac5c92f0a5817a82 (DEFAULT_TABLE table=test-table [id=1d5108af036c48a899022a88ddb60f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:18.769208 7302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 458d8759c261420aac5c92f0a5817a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:18.779734 7670 tablet_bootstrap.cc:492] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Bootstrap starting.
I20260923 20:46:18.783990 7670 tablet_bootstrap.cc:654] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:18.784838 7671 tablet_bootstrap.cc:492] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Bootstrap starting.
I20260923 20:46:18.785378 7670 log.cc:824] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:18.786933 7672 tablet_bootstrap.cc:492] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Bootstrap starting.
I20260923 20:46:18.788550 7671 tablet_bootstrap.cc:654] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:18.788787 7670 tablet_bootstrap.cc:492] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: No bootstrap required, opened a new log
I20260923 20:46:18.789076 7670 ts_tablet_manager.cc:1397] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260923 20:46:18.789846 7671 log.cc:824] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:18.790475 7672 tablet_bootstrap.cc:654] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:18.792107 7672 log.cc:824] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:18.793125 7671 tablet_bootstrap.cc:492] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: No bootstrap required, opened a new log
I20260923 20:46:18.793431 7671 ts_tablet_manager.cc:1397] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260923 20:46:18.794560 7672 tablet_bootstrap.cc:492] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: No bootstrap required, opened a new log
I20260923 20:46:18.794807 7672 ts_tablet_manager.cc:1397] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260923 20:46:18.800822 7670 raft_consensus.cc:348] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:18.801329 7670 raft_consensus.cc:374] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:18.801489 7670 raft_consensus.cc:729] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Initialized, Role: FOLLOWER
I20260923 20:46:18.802217 7670 consensus_queue.cc:260] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:18.802248 7672 raft_consensus.cc:348] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:18.802717 7672 raft_consensus.cc:374] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:18.802829 7672 raft_consensus.cc:729] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5addcbb4bcd24b5aafd90f7d737a2f49, State: Initialized, Role: FOLLOWER
I20260923 20:46:18.803921 7651 heartbeater.cc:503] Master 127.5.71.190:44639 was elected leader, sending a full tablet report...
I20260923 20:46:18.804713 7670 ts_tablet_manager.cc:1428] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260923 20:46:18.803618 7672 consensus_queue.cc:260] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:18.805079 7671 raft_consensus.cc:348] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:18.805593 7671 raft_consensus.cc:374] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:18.805815 7671 raft_consensus.cc:729] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Initialized, Role: FOLLOWER
I20260923 20:46:18.806509 7671 consensus_queue.cc:260] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:18.807637 7511 heartbeater.cc:503] Master 127.5.71.190:44639 was elected leader, sending a full tablet report...
I20260923 20:46:18.809417 7672 ts_tablet_manager.cc:1428] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260923 20:46:18.809799 7671 ts_tablet_manager.cc:1428] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260923 20:46:18.850739 7512 tablet.cc:2369] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:18.908382 7652 tablet.cc:2369] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:19.028554 7678 raft_consensus.cc:482] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:19.028990 7678 raft_consensus.cc:504] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:19.030299 7676 raft_consensus.cc:482] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:19.030652 7676 raft_consensus.cc:504] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:19.031062 7678 leader_election.cc:288] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379), 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049)
I20260923 20:46:19.032829 7676 leader_election.cc:288] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:19.041265 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.032383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:55119 (local address 127.5.71.129:39049)
0923 20:46:19.033565 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.033572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.033589 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:46:19.033658 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.033663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.033753 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.033841 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:19.034728 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.036010 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.038797 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.040056 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.040190 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.040276 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.040844 (+ 568us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.040987 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1037}
I20260923 20:46:19.041368 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.032088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.5.71.130:55119)
0923 20:46:19.032983 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.033001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.033119 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.034006 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.034010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.034033 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:19.034486 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.034503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.036169 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.036177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.038570 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.038584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.040474 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.040482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.040666 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.040700 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.040813 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.040956 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":5,"thread_start_us":519,"threads_started":1}
I20260923 20:46:19.043427 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.033087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.5.71.131:47073)
0923 20:46:19.033321 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.033340 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.033441 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.036208 (+ 2767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.036215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.036249 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:19.036528 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.036540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.038356 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.038362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.040936 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.040951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.042553 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.042559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.042826 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.042860 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.042953 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.043105 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":109}
I20260923 20:46:19.043426 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.031631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:37903 (local address 127.5.71.131:41379)
0923 20:46:19.031873 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.031885 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.032109 (+ 224us) server_negotiation.cc:407] Connection header received
0923 20:46:19.032367 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.032373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.032447 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.032512 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:19.033351 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.034739 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.040106 (+ 5367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.042347 (+ 2241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.042528 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.042614 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.042973 (+ 359us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.043122 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":913,"server-negotiator.queue_time_us":62}
I20260923 20:46:19.043869 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.034031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:50599 (local address 127.5.71.130:38519)
0923 20:46:19.034210 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.034215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.035447 (+ 1232us) server_negotiation.cc:407] Connection header received
0923 20:46:19.035711 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.035716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.035806 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.035888 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:19.036713 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.037962 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.041796 (+ 3834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.042676 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.042829 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.042903 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.043425 (+ 522us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.043583 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260923 20:46:19.043764 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" is_pre_election: true
I20260923 20:46:19.044373 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.031272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.5.71.130:37903)
0923 20:46:19.031990 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.032011 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.032172 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.032733 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.032742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.032776 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:19.033180 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.033195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.035918 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.035925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.039935 (+ 4010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.039951 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.042592 (+ 2641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.042600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.042784 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.042825 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.043456 (+ 631us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.043721 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":77,"thread_start_us":314,"threads_started":1}
I20260923 20:46:19.044486 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.033659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.5.71.131:50599)
0923 20:46:19.035347 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.035370 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.035552 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.036029 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.036034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.036063 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:19.036531 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.036546 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.038174 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.038179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.041584 (+ 3405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.041601 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.043059 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.043065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.043241 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.043277 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.043936 (+ 659us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.044113 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1550,"mutex_wait_us":544,"thread_start_us":1385,"threads_started":1}
I20260923 20:46:19.044535 7322 raft_consensus.cc:2454] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 0.
I20260923 20:46:19.045050 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.033769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:47073 (local address 127.5.71.129:39049)
0923 20:46:19.035872 (+ 2103us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.035877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.035899 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:46:19.035960 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.035966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.036044 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.036676 (+ 632us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:19.036945 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.038197 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.041131 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.042409 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.042664 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.042732 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.044264 (+ 1532us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.044415 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1923,"thread_start_us":1620,"threads_started":1}
I20260923 20:46:19.045320 7394 leader_election.cc:302] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d59002757c647cc95560b1c8fe5710b, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:19.045715 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" is_pre_election: true
I20260923 20:46:19.045812 7678 raft_consensus.cc:2781] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:19.045747 7606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329d0db9e44c4661989fed81051123fa" is_pre_election: true
I20260923 20:46:19.045974 7678 raft_consensus.cc:482] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:19.045966 7322 raft_consensus.cc:2454] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 0.
I20260923 20:46:19.046080 7678 raft_consensus.cc:3037] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.046176 7606 raft_consensus.cc:2454] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 0.
I20260923 20:46:19.046357 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b" is_pre_election: true
I20260923 20:46:19.046671 7533 leader_election.cc:302] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:19.047066 7676 raft_consensus.cc:2781] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:19.047209 7676 raft_consensus.cc:482] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:19.047289 7676 raft_consensus.cc:3037] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Advancing to term 1
W20260923 20:46:19.048271 7373 tablet.cc:2369] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:19.049309 7678 raft_consensus.cc:504] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:19.049822 7466 raft_consensus.cc:2379] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 329d0db9e44c4661989fed81051123fa in current term 1: Already voted for candidate 4d59002757c647cc95560b1c8fe5710b in this term.
I20260923 20:46:19.050140 7678 leader_election.cc:288] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 election: Requested vote from peers 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379), 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049)
I20260923 20:46:19.050195 7676 raft_consensus.cc:504] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:19.050738 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
I20260923 20:46:19.050897 7676 leader_election.cc:288] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 election: Requested vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:19.050973 7322 raft_consensus.cc:3037] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.051496 7321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
I20260923 20:46:19.051693 7606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329d0db9e44c4661989fed81051123fa"
I20260923 20:46:19.052013 7606 raft_consensus.cc:2379] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d59002757c647cc95560b1c8fe5710b in current term 1: Already voted for candidate 329d0db9e44c4661989fed81051123fa in this term.
I20260923 20:46:19.052630 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "458d8759c261420aac5c92f0a5817a82" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b"
I20260923 20:46:19.052884 7466 raft_consensus.cc:2379] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 329d0db9e44c4661989fed81051123fa in current term 1: Already voted for candidate 4d59002757c647cc95560b1c8fe5710b in this term.
I20260923 20:46:19.053253 7534 leader_election.cc:302] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 329d0db9e44c4661989fed81051123fa; no voters: 4d59002757c647cc95560b1c8fe5710b, 5addcbb4bcd24b5aafd90f7d737a2f49
I20260923 20:46:19.053535 7676 raft_consensus.cc:2726] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260923 20:46:19.054194 7322 raft_consensus.cc:2454] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 1.
I20260923 20:46:19.054536 7394 leader_election.cc:302] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d59002757c647cc95560b1c8fe5710b, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters: 329d0db9e44c4661989fed81051123fa
I20260923 20:46:19.054971 7678 raft_consensus.cc:2781] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:19.055438 7678 raft_consensus.cc:686] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 LEADER]: Becoming Leader. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Running, Role: LEADER
I20260923 20:46:19.055986 7678 consensus_queue.cc:237] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } }
I20260923 20:46:19.061172 7164 catalog_manager.cc:5495] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d59002757c647cc95560b1c8fe5710b (127.5.71.130). New cstate: current_term: 1 leader_uuid: "4d59002757c647cc95560b1c8fe5710b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } health_report { overall_health: HEALTHY } } }
Sep 23 20:46:19 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196376, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:19.157202 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.142572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44639 (local address 127.0.0.1:42588)
0923 20:46:19.143253 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.143270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.143419 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.143735 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.143845 (+ 110us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.144651 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:19.144956 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.144965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.146362 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.146367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.147323 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.147334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.148283 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.148288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.148411 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.149064 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:19.149085 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:19.151374 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:19.154831 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:19.154842 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:19.154848 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:19.155243 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:19.155599 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:19.155603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:19.155605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:19.155737 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:19.156334 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:19.156340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:19.156492 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.156626 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.156872 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":405,"threads_started":1}
I20260923 20:46:19.157366 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.142504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.5.71.190:44639)
0923 20:46:19.142712 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.142722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.143380 (+ 658us) server_negotiation.cc:407] Connection header received
0923 20:46:19.143527 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.143531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.143592 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.143677 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:19.145108 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.146182 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.147475 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.148082 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.148212 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.151576 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:19.151581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:19.151584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:19.151616 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:19.154674 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:19.155365 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:19.155370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:19.155372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:19.155500 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:19.155949 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:19.155953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:19.155955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:19.156213 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:19.156338 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.156707 (+ 369us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.157009 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":71}
I20260923 20:46:19.162719 7164 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:42588:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 20:46:19.163760 7164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:19.170616 7441 tablet_service.cc:1467] Processing CreateTablet for tablet cef7d696f03745339dc8a85446c4b93e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fa86b0977a6472eaee6ba802c3a7f60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:46:19.170411 7302 tablet_service.cc:1467] Processing CreateTablet for tablet cef7d696f03745339dc8a85446c4b93e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fa86b0977a6472eaee6ba802c3a7f60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:46:19.170851 7581 tablet_service.cc:1467] Processing CreateTablet for tablet cef7d696f03745339dc8a85446c4b93e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fa86b0977a6472eaee6ba802c3a7f60]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:46:19.171506 7441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef7d696f03745339dc8a85446c4b93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:19.171506 7302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef7d696f03745339dc8a85446c4b93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:19.171754 7581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cef7d696f03745339dc8a85446c4b93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:19.177445 7670 tablet_bootstrap.cc:492] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: Bootstrap starting.
I20260923 20:46:19.178630 7671 tablet_bootstrap.cc:492] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: Bootstrap starting.
I20260923 20:46:19.178617 7672 tablet_bootstrap.cc:492] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: Bootstrap starting.
I20260923 20:46:19.178838 7670 tablet_bootstrap.cc:654] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:19.180119 7671 tablet_bootstrap.cc:654] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:19.180442 7672 tablet_bootstrap.cc:654] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:19.182956 7671 tablet_bootstrap.cc:492] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: No bootstrap required, opened a new log
I20260923 20:46:19.183131 7671 ts_tablet_manager.cc:1397] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 20:46:19.183310 7672 tablet_bootstrap.cc:492] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: No bootstrap required, opened a new log
I20260923 20:46:19.183472 7672 ts_tablet_manager.cc:1397] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 20:46:19.183915 7671 raft_consensus.cc:348] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.184115 7671 raft_consensus.cc:374] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:19.184216 7671 raft_consensus.cc:729] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Initialized, Role: FOLLOWER
I20260923 20:46:19.184360 7670 tablet_bootstrap.cc:492] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: No bootstrap required, opened a new log
I20260923 20:46:19.184280 7672 raft_consensus.cc:348] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.184500 7670 ts_tablet_manager.cc:1397] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260923 20:46:19.184542 7672 raft_consensus.cc:374] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:19.184464 7671 consensus_queue.cc:260] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.184651 7672 raft_consensus.cc:729] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5addcbb4bcd24b5aafd90f7d737a2f49, State: Initialized, Role: FOLLOWER
I20260923 20:46:19.184861 7672 consensus_queue.cc:260] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.184897 7678 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.185168 7678 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.185256 7670 raft_consensus.cc:348] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.185425 7672 ts_tablet_manager.cc:1428] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 20:46:19.185498 7670 raft_consensus.cc:374] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:19.185587 7670 raft_consensus.cc:729] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Initialized, Role: FOLLOWER
I20260923 20:46:19.185801 7671 ts_tablet_manager.cc:1428] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:19.185810 7670 consensus_queue.cc:260] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.186373 7670 ts_tablet_manager.cc:1428] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260923 20:46:19.186260 7676 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.186550 7676 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.186358 7677 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.186690 7677 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.446339 7678 consensus_queue.cc:1035] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [LEADER]: Connected to new peer: Peer: permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:19.455500 7683 consensus_queue.cc:1035] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:19.545116 7676 raft_consensus.cc:482] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:19.545331 7676 raft_consensus.cc:504] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.545996 7676 leader_election.cc:288] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:19.546649 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7d696f03745339dc8a85446c4b93e" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b" is_pre_election: true
I20260923 20:46:19.546921 7466 raft_consensus.cc:2454] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 0.
I20260923 20:46:19.546787 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7d696f03745339dc8a85446c4b93e" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" is_pre_election: true
I20260923 20:46:19.547483 7534 leader_election.cc:302] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 4d59002757c647cc95560b1c8fe5710b; no voters:
I20260923 20:46:19.547770 7676 raft_consensus.cc:2781] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:19.547910 7676 raft_consensus.cc:482] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:19.547914 7322 raft_consensus.cc:2454] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 0.
I20260923 20:46:19.548000 7676 raft_consensus.cc:3037] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.550851 7676 raft_consensus.cc:504] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.551551 7676 leader_election.cc:288] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 election: Requested vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:19.552275 7322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7d696f03745339dc8a85446c4b93e" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
I20260923 20:46:19.552248 7466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef7d696f03745339dc8a85446c4b93e" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b"
I20260923 20:46:19.552459 7466 raft_consensus.cc:3037] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.552520 7322 raft_consensus.cc:3037] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.554649 7322 raft_consensus.cc:2454] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 1.
I20260923 20:46:19.555150 7533 leader_election.cc:302] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:19.555397 7466 raft_consensus.cc:2454] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 1.
I20260923 20:46:19.555451 7676 raft_consensus.cc:2781] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:19.556066 7676 raft_consensus.cc:686] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [term 1 LEADER]: Becoming Leader. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Running, Role: LEADER
I20260923 20:46:19.556558 7676 consensus_queue.cc:237] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.559471 7700 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: TxnStatusTablet state changed. Reason: New leader 329d0db9e44c4661989fed81051123fa. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.559731 7700 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:46:19.560415 7702 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 20:46:19.560747 7702 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 20:46:19.560570 7165 catalog_manager.cc:5495] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 329d0db9e44c4661989fed81051123fa (127.5.71.131). New cstate: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } health_report { overall_health: HEALTHY } } }
I20260923 20:46:19.642361 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.635321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39792 (local address 127.5.71.131:41379)
0923 20:46:19.636000 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.636005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.636026 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:46:19.636115 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.636118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.636202 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.636344 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:19.636796 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.637694 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.639645 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.640204 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.640307 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.640889 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.641565 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.641710 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.641928 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.642051 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":415,"threads_started":1}
I20260923 20:46:19.642393 7690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.635139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.0.0.1:39792)
0923 20:46:19.635377 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.635397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.635551 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.636403 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.636409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.636423 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:19.636621 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.636631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.637834 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.637840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.639499 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.639517 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.640367 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.640374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.640555 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.640631 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.641686 (+ 1055us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.641712 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.641838 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.642012 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 20:46:19.650754 7322 raft_consensus.cc:1261] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Refusing update from remote peer 329d0db9e44c4661989fed81051123fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:19.650970 7466 raft_consensus.cc:1261] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Refusing update from remote peer 329d0db9e44c4661989fed81051123fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:19.651655 7676 consensus_queue.cc:1035] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:19.652076 7700 consensus_queue.cc:1035] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:19.667637 7683 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: TxnStatusTablet state changed. Reason: New leader 329d0db9e44c4661989fed81051123fa. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.668563 7683 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.671928 7695 mvcc.cc:205] Tried to move back new op lower bound from 7332644371041181696 to 7332644370667548672. Current Snapshot: MvccSnapshot[applied={T|T < 7332644371041181696}]
I20260923 20:46:19.676498 7700 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.676802 7700 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:46:19.678475 7683 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.678800 7683 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 4d59002757c647cc95560b1c8fe5710b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.685017 7677 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: TxnStatusTablet state changed. Reason: New leader 329d0db9e44c4661989fed81051123fa. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.685320 7677 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.688651 7676 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.689410 7676 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 329d0db9e44c4661989fed81051123fa: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:46:19.712996 7677 tablet_replica.cc:439] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } } }
I20260923 20:46:19.713320 7677 tablet_replica.cc:442] T cef7d696f03745339dc8a85446c4b93e P 5addcbb4bcd24b5aafd90f7d737a2f49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:46:19.720598 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.701916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.5.71.130:38519)
0923 20:46:19.711170 (+ 9254us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.711178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.711206 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:46:19.711366 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.711373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.711514 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.711727 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:19.714438 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.715700 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.717418 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.718201 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.718346 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.718889 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.719712 (+ 823us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.719912 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.720120 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.720221 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9103,"thread_start_us":8999,"threads_started":1}
I20260923 20:46:19.738533 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.732968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:32969 (local address 127.5.71.131:41379)
0923 20:46:19.733190 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.733202 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.733517 (+ 315us) server_negotiation.cc:407] Connection header received
0923 20:46:19.733745 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.733755 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.733820 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.733916 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:19.734391 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.735167 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.736680 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.737123 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.737261 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.737757 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.737887 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.737995 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.738130 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.738214 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260923 20:46:19.738631 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.732786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.5.71.130:32969)
0923 20:46:19.733407 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.733422 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.733605 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.733945 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.733950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.733963 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:19.734233 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.734249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.735308 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.735317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.736539 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.736551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.737300 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.737307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.737443 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.737502 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.737993 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.738007 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.738108 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.738253 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":283,"threads_started":1}
I20260923 20:46:19.779294 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.772304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:45599 (local address 127.5.71.130:38519)
0923 20:46:19.772461 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:19.772466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:19.773055 (+ 589us) server_negotiation.cc:407] Connection header received
0923 20:46:19.773279 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:19.773285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:19.773362 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:19.773460 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:19.774002 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.775119 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.776923 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.777597 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.777746 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.778277 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.778411 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.778534 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:19.778897 (+ 363us) server_negotiation.cc:299] Negotiation successful
0923 20:46:19.778978 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 20:46:19.779445 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:19.772332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.5.71.131:45599)
0923 20:46:19.772945 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:19.772961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:19.773128 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:19.773474 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:19.773479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:19.773489 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:19.773802 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.773834 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.775365 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.775369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.776749 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:19.776761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:19.777802 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:19.777810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:19.777978 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:19.778056 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:19.778757 (+ 701us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:19.778767 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 20:46:19.778879 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 20:46:19.779031 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":315,"threads_started":1}
I20260923 20:46:19.822906 7165 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:19.823261 7165 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:19.828116 7165 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Sending DeleteTablet for 3 replicas of tablet 458d8759c261420aac5c92f0a5817a82
I20260923 20:46:19.829681 7441 tablet_service.cc:1514] Processing DeleteTablet for tablet 458d8759c261420aac5c92f0a5817a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:19 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:34318
I20260923 20:46:19.829826 7302 tablet_service.cc:1514] Processing DeleteTablet for tablet 458d8759c261420aac5c92f0a5817a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:19 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57068
I20260923 20:46:19.830713 7581 tablet_service.cc:1514] Processing DeleteTablet for tablet 458d8759c261420aac5c92f0a5817a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:19 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:39784
I20260923 20:46:19.832779 7164 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:19.833577 7164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:19.843276 7580 tablet_service.cc:1467] Processing CreateTablet for tablet 4b46ab1782e2404da8629cdaaa3ddc22 (DEFAULT_TABLE table=test-table [id=e6c08d074c6e4ce58a9514697e81187e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:19.844075 7580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b46ab1782e2404da8629cdaaa3ddc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:19.853075 7440 tablet_service.cc:1467] Processing CreateTablet for tablet 4b46ab1782e2404da8629cdaaa3ddc22 (DEFAULT_TABLE table=test-table [id=e6c08d074c6e4ce58a9514697e81187e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:19.853816 7440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b46ab1782e2404da8629cdaaa3ddc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:19.856365 7301 tablet_service.cc:1467] Processing CreateTablet for tablet 4b46ab1782e2404da8629cdaaa3ddc22 (DEFAULT_TABLE table=test-table [id=e6c08d074c6e4ce58a9514697e81187e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:19.856927 7301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b46ab1782e2404da8629cdaaa3ddc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:19.876220 7721 tablet_replica.cc:330] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: stopping tablet replica
I20260923 20:46:19.876788 7721 raft_consensus.cc:2229] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:19.878850 7725 tablet_bootstrap.cc:492] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Bootstrap starting.
I20260923 20:46:19.879106 7721 raft_consensus.cc:2258] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:19.881345 7725 tablet_bootstrap.cc:654] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:19.881626 7721 ts_tablet_manager.cc:1905] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:19.882273 7726 tablet_bootstrap.cc:492] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Bootstrap starting.
I20260923 20:46:19.886279 7723 tablet_replica.cc:330] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: stopping tablet replica
I20260923 20:46:19.886852 7723 raft_consensus.cc:2229] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:19.887300 7723 raft_consensus.cc:2258] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:19.889850 7723 ts_tablet_manager.cc:1905] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:19.890461 7722 tablet_replica.cc:330] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: stopping tablet replica
I20260923 20:46:19.890970 7722 raft_consensus.cc:2229] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:19.891440 7722 raft_consensus.cc:2258] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:19.893625 7722 ts_tablet_manager.cc:1905] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:19.894467 7726 tablet_bootstrap.cc:654] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:19.902225 7721 ts_tablet_manager.cc:1918] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:19.902328 7725 tablet_bootstrap.cc:492] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: No bootstrap required, opened a new log
I20260923 20:46:19.902448 7721 log.cc:1196] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/458d8759c261420aac5c92f0a5817a82
I20260923 20:46:19.902717 7725 ts_tablet_manager.cc:1397] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260923 20:46:19.895074 7724 tablet_bootstrap.cc:492] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Bootstrap starting.
I20260923 20:46:19.903128 7721 ts_tablet_manager.cc:1939] T 458d8759c261420aac5c92f0a5817a82 P 4d59002757c647cc95560b1c8fe5710b: Deleting consensus metadata
I20260923 20:46:19.906910 7725 raft_consensus.cc:348] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.907240 7725 raft_consensus.cc:374] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:19.907349 7725 raft_consensus.cc:729] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Initialized, Role: FOLLOWER
I20260923 20:46:19.907622 7725 consensus_queue.cc:260] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.908547 7725 ts_tablet_manager.cc:1428] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 20:46:19.909355 7152 catalog_manager.cc:4841] TS 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519): tablet 458d8759c261420aac5c92f0a5817a82 (table test-table [id=1d5108af036c48a899022a88ddb60f25]) successfully deleted
I20260923 20:46:19.913671 7726 tablet_bootstrap.cc:492] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: No bootstrap required, opened a new log
I20260923 20:46:19.913760 7722 ts_tablet_manager.cc:1918] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:19.913874 7726 ts_tablet_manager.cc:1397] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.000s
I20260923 20:46:19.913944 7722 log.cc:1196] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/458d8759c261420aac5c92f0a5817a82
I20260923 20:46:19.914588 7722 ts_tablet_manager.cc:1939] T 458d8759c261420aac5c92f0a5817a82 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting consensus metadata
I20260923 20:46:19.914847 7726 raft_consensus.cc:348] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.915138 7726 raft_consensus.cc:374] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:19.915256 7726 raft_consensus.cc:729] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5addcbb4bcd24b5aafd90f7d737a2f49, State: Initialized, Role: FOLLOWER
I20260923 20:46:19.915539 7726 consensus_queue.cc:260] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.916270 7726 ts_tablet_manager.cc:1428] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 20:46:19.916888 7151 catalog_manager.cc:4841] TS 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049): tablet 458d8759c261420aac5c92f0a5817a82 (table test-table [id=1d5108af036c48a899022a88ddb60f25]) successfully deleted
I20260923 20:46:19.920856 7724 tablet_bootstrap.cc:654] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:19.927409 7723 ts_tablet_manager.cc:1918] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:19.927608 7723 log.cc:1196] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/458d8759c261420aac5c92f0a5817a82
I20260923 20:46:19.928247 7723 ts_tablet_manager.cc:1939] T 458d8759c261420aac5c92f0a5817a82 P 329d0db9e44c4661989fed81051123fa: Deleting consensus metadata
I20260923 20:46:19.930378 7151 catalog_manager.cc:4841] TS 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379): tablet 458d8759c261420aac5c92f0a5817a82 (table test-table [id=1d5108af036c48a899022a88ddb60f25]) successfully deleted
I20260923 20:46:19.936533 7724 tablet_bootstrap.cc:492] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: No bootstrap required, opened a new log
I20260923 20:46:19.936767 7724 ts_tablet_manager.cc:1397] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.000s
I20260923 20:46:19.937574 7724 raft_consensus.cc:348] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.937865 7724 raft_consensus.cc:374] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:19.937994 7724 raft_consensus.cc:729] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Initialized, Role: FOLLOWER
I20260923 20:46:19.938289 7724 consensus_queue.cc:260] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.939122 7724 ts_tablet_manager.cc:1428] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 20:46:19.945632 7700 raft_consensus.cc:482] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:19.945883 7700 raft_consensus.cc:504] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.946597 7700 leader_election.cc:288] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:19.947085 7321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b46ab1782e2404da8629cdaaa3ddc22" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" is_pre_election: true
I20260923 20:46:19.947381 7321 raft_consensus.cc:2454] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 0.
I20260923 20:46:19.947409 7465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b46ab1782e2404da8629cdaaa3ddc22" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b" is_pre_election: true
I20260923 20:46:19.947705 7465 raft_consensus.cc:2454] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 0.
I20260923 20:46:19.948149 7533 leader_election.cc:302] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:19.948495 7700 raft_consensus.cc:2781] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:19.948633 7700 raft_consensus.cc:482] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:19.948762 7700 raft_consensus.cc:3037] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.951555 7700 raft_consensus.cc:504] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.952634 7321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b46ab1782e2404da8629cdaaa3ddc22" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
I20260923 20:46:19.952669 7465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b46ab1782e2404da8629cdaaa3ddc22" candidate_uuid: "329d0db9e44c4661989fed81051123fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b"
I20260923 20:46:19.952888 7465 raft_consensus.cc:3037] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.952899 7321 raft_consensus.cc:3037] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:19.956128 7321 raft_consensus.cc:2454] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 1.
I20260923 20:46:19.956164 7465 raft_consensus.cc:2454] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 329d0db9e44c4661989fed81051123fa in term 1.
I20260923 20:46:19.956687 7534 leader_election.cc:302] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 4d59002757c647cc95560b1c8fe5710b, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:19.957126 7676 raft_consensus.cc:2781] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:19.957156 7700 leader_election.cc:288] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [CANDIDATE]: Term 1 election: Requested vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519)
I20260923 20:46:19.969555 7676 raft_consensus.cc:686] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 1 LEADER]: Becoming Leader. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Running, Role: LEADER
I20260923 20:46:19.970014 7676 consensus_queue.cc:237] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:19.973165 7164 catalog_manager.cc:5495] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 329d0db9e44c4661989fed81051123fa (127.5.71.131). New cstate: current_term: 1 leader_uuid: "329d0db9e44c4661989fed81051123fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } health_report { overall_health: HEALTHY } } }
I20260923 20:46:20.016340 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:20.010490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.5.71.131:41379)
0923 20:46:20.010659 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:20.010665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:20.010803 (+ 138us) server_negotiation.cc:407] Connection header received
0923 20:46:20.011057 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:20.011070 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:20.011150 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:20.011291 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:20.011869 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:20.012678 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:20.014193 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:20.014817 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:20.014984 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:20.015448 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:20.015561 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:20.015670 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:20.015856 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 20:46:20.015943 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 20:46:20.031095 7705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:20.026233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:40511 (local address 127.5.71.131:41379)
0923 20:46:20.026407 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:20.026412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:20.026514 (+ 102us) server_negotiation.cc:407] Connection header received
0923 20:46:20.026703 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:20.026707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:20.026765 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:20.026868 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:20.027302 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:20.028082 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:20.029346 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:20.029837 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:20.029977 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:20.030350 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:20.030465 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:20.030566 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:20.030665 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 20:46:20.030753 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":44}
I20260923 20:46:20.031102 7718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:20.026200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.5.71.131:40511)
0923 20:46:20.026412 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:20.026438 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:20.026580 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:20.026877 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:20.026881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:20.026891 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:20.027126 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:20.027135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:20.028203 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:20.028209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:20.029207 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:20.029217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:20.029995 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:20.030001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:20.030160 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:20.030193 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:20.030565 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:20.030575 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 20:46:20.030651 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 20:46:20.030802 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":34}
I20260923 20:46:20.042796 7321 raft_consensus.cc:1261] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Refusing update from remote peer 329d0db9e44c4661989fed81051123fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:20.042893 7465 raft_consensus.cc:1261] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Refusing update from remote peer 329d0db9e44c4661989fed81051123fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:20.043540 7730 consensus_queue.cc:1035] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:20.044317 7730 consensus_queue.cc:1035] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:20.058022 7697 mvcc.cc:205] Tried to move back new op lower bound from 7332644372649275392 to 7332644372360491008. Current Snapshot: MvccSnapshot[applied={T|T < 7332644372649275392}]
W20260923 20:46:20.085731 7697 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 20:46:21.710649 7164 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:21.710912 7164 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:21.716195 7164 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Sending DeleteTablet for 3 replicas of tablet 4b46ab1782e2404da8629cdaaa3ddc22
I20260923 20:46:21.717476 7580 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b46ab1782e2404da8629cdaaa3ddc22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:21 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:39784
I20260923 20:46:21.717515 7440 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b46ab1782e2404da8629cdaaa3ddc22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:21 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:34318
I20260923 20:46:21.717567 7302 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b46ab1782e2404da8629cdaaa3ddc22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:21 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57068
I20260923 20:46:21.728909 7756 tablet_replica.cc:330] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: stopping tablet replica
I20260923 20:46:21.729302 7756 raft_consensus.cc:2229] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:21.730201 7756 raft_consensus.cc:2258] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:21.731070 7755 tablet_replica.cc:330] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: stopping tablet replica
I20260923 20:46:21.731410 7755 raft_consensus.cc:2229] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:21.731580 7757 tablet_replica.cc:330] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: stopping tablet replica
I20260923 20:46:21.731887 7755 raft_consensus.cc:2258] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:21.731925 7757 raft_consensus.cc:2229] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:21.732393 7757 raft_consensus.cc:2258] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:21.733137 7755 ts_tablet_manager.cc:1905] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:21.733398 7757 ts_tablet_manager.cc:1905] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:21.733353 7756 ts_tablet_manager.cc:1905] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:21.742090 7755 ts_tablet_manager.cc:1918] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 20:46:21.742090 7757 ts_tablet_manager.cc:1918] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 20:46:21.742308 7757 log.cc:1196] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/4b46ab1782e2404da8629cdaaa3ddc22
I20260923 20:46:21.742316 7755 log.cc:1196] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/4b46ab1782e2404da8629cdaaa3ddc22
I20260923 20:46:21.742853 7756 ts_tablet_manager.cc:1918] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 20:46:21.742981 7755 ts_tablet_manager.cc:1939] T 4b46ab1782e2404da8629cdaaa3ddc22 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting consensus metadata
I20260923 20:46:21.743076 7757 ts_tablet_manager.cc:1939] T 4b46ab1782e2404da8629cdaaa3ddc22 P 4d59002757c647cc95560b1c8fe5710b: Deleting consensus metadata
I20260923 20:46:21.743219 7756 log.cc:1196] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/4b46ab1782e2404da8629cdaaa3ddc22
I20260923 20:46:21.744228 7756 ts_tablet_manager.cc:1939] T 4b46ab1782e2404da8629cdaaa3ddc22 P 329d0db9e44c4661989fed81051123fa: Deleting consensus metadata
I20260923 20:46:21.744319 7152 catalog_manager.cc:4841] TS 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519): tablet 4b46ab1782e2404da8629cdaaa3ddc22 (table test-table [id=e6c08d074c6e4ce58a9514697e81187e]) successfully deleted
I20260923 20:46:21.744414 7151 catalog_manager.cc:4841] TS 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049): tablet 4b46ab1782e2404da8629cdaaa3ddc22 (table test-table [id=e6c08d074c6e4ce58a9514697e81187e]) successfully deleted
I20260923 20:46:21.745513 7151 catalog_manager.cc:4841] TS 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379): tablet 4b46ab1782e2404da8629cdaaa3ddc22 (table test-table [id=e6c08d074c6e4ce58a9514697e81187e]) successfully deleted
Sep 23 20:46:21 dist-test-slave-dn29 krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 20:46:21 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:21.788847 7764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:21.772118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.5.71.190:44639)
0923 20:46:21.772915 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:21.772919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:21.772943 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:46:21.773214 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:21.773219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:21.773289 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:21.773422 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:21.774968 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:21.776010 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:21.777401 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:21.777836 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:21.777939 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:21.783520 (+ 5581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:21.783526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:21.783530 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:21.783567 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:21.786182 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:21.786845 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:21.786849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:21.786851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:21.786968 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:21.787378 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:21.787381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:21.787383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:21.787648 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:21.787926 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:21.788186 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 20:46:21.788355 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":539,"threads_started":1}
I20260923 20:46:21.792644 7164 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:21.793210 7164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:21.800225 7302 tablet_service.cc:1467] Processing CreateTablet for tablet 5b748ef7cf9249bdb023e384c6f17144 (DEFAULT_TABLE table=test-table [id=71ef76737caf4354808b53a5bb7305dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:21.800220 7440 tablet_service.cc:1467] Processing CreateTablet for tablet 5b748ef7cf9249bdb023e384c6f17144 (DEFAULT_TABLE table=test-table [id=71ef76737caf4354808b53a5bb7305dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:21.800663 7302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b748ef7cf9249bdb023e384c6f17144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:21.800683 7440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b748ef7cf9249bdb023e384c6f17144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:21.801084 7580 tablet_service.cc:1467] Processing CreateTablet for tablet 5b748ef7cf9249bdb023e384c6f17144 (DEFAULT_TABLE table=test-table [id=71ef76737caf4354808b53a5bb7305dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:21.801579 7580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b748ef7cf9249bdb023e384c6f17144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:21.808398 7766 tablet_bootstrap.cc:492] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Bootstrap starting.
I20260923 20:46:21.808579 7767 tablet_bootstrap.cc:492] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Bootstrap starting.
I20260923 20:46:21.808746 7768 tablet_bootstrap.cc:492] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Bootstrap starting.
I20260923 20:46:21.810277 7766 tablet_bootstrap.cc:654] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:21.810422 7768 tablet_bootstrap.cc:654] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:21.810479 7767 tablet_bootstrap.cc:654] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:21.816066 7767 tablet_bootstrap.cc:492] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: No bootstrap required, opened a new log
I20260923 20:46:21.816221 7768 tablet_bootstrap.cc:492] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: No bootstrap required, opened a new log
I20260923 20:46:21.816246 7767 ts_tablet_manager.cc:1397] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260923 20:46:21.816308 7766 tablet_bootstrap.cc:492] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: No bootstrap required, opened a new log
I20260923 20:46:21.816449 7766 ts_tablet_manager.cc:1397] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260923 20:46:21.816591 7768 ts_tablet_manager.cc:1397] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260923 20:46:21.817121 7767 raft_consensus.cc:348] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.817386 7767 raft_consensus.cc:374] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:21.817253 7766 raft_consensus.cc:348] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.817493 7767 raft_consensus.cc:729] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Initialized, Role: FOLLOWER
I20260923 20:46:21.817521 7766 raft_consensus.cc:374] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:21.817642 7766 raft_consensus.cc:729] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5addcbb4bcd24b5aafd90f7d737a2f49, State: Initialized, Role: FOLLOWER
I20260923 20:46:21.817525 7768 raft_consensus.cc:348] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.817755 7768 raft_consensus.cc:374] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:21.817860 7768 raft_consensus.cc:729] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Initialized, Role: FOLLOWER
I20260923 20:46:21.817781 7767 consensus_queue.cc:260] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.817945 7766 consensus_queue.cc:260] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.818099 7768 consensus_queue.cc:260] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.818531 7767 ts_tablet_manager.cc:1428] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260923 20:46:21.819427 7766 ts_tablet_manager.cc:1428] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 20:46:21.820202 7768 ts_tablet_manager.cc:1428] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:21.972965 7773 raft_consensus.cc:482] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:21.973217 7773 raft_consensus.cc:504] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.974021 7773 leader_election.cc:288] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379)
I20260923 20:46:21.974725 7321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b748ef7cf9249bdb023e384c6f17144" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" is_pre_election: true
I20260923 20:46:21.974748 7606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b748ef7cf9249bdb023e384c6f17144" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329d0db9e44c4661989fed81051123fa" is_pre_election: true
I20260923 20:46:21.975014 7606 raft_consensus.cc:2454] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 0.
I20260923 20:46:21.975049 7321 raft_consensus.cc:2454] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 0.
I20260923 20:46:21.975536 7394 leader_election.cc:302] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 4d59002757c647cc95560b1c8fe5710b; no voters:
I20260923 20:46:21.975821 7773 raft_consensus.cc:2781] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:21.975975 7773 raft_consensus.cc:482] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:21.976079 7773 raft_consensus.cc:3037] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:21.978084 7773 raft_consensus.cc:504] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.978603 7773 leader_election.cc:288] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 election: Requested vote from peers 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049), 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379)
I20260923 20:46:21.979230 7321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b748ef7cf9249bdb023e384c6f17144" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49"
I20260923 20:46:21.979271 7606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b748ef7cf9249bdb023e384c6f17144" candidate_uuid: "4d59002757c647cc95560b1c8fe5710b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329d0db9e44c4661989fed81051123fa"
I20260923 20:46:21.979473 7606 raft_consensus.cc:3037] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:21.979475 7321 raft_consensus.cc:3037] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:21.981670 7321 raft_consensus.cc:2454] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 1.
I20260923 20:46:21.981912 7606 raft_consensus.cc:2454] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d59002757c647cc95560b1c8fe5710b in term 1.
I20260923 20:46:21.982261 7394 leader_election.cc:302] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d59002757c647cc95560b1c8fe5710b, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:21.982740 7773 raft_consensus.cc:2781] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:21.983188 7773 raft_consensus.cc:686] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 1 LEADER]: Becoming Leader. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Running, Role: LEADER
I20260923 20:46:21.983476 7773 consensus_queue.cc:237] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:21.986141 7164 catalog_manager.cc:5495] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d59002757c647cc95560b1c8fe5710b (127.5.71.130). New cstate: current_term: 1 leader_uuid: "4d59002757c647cc95560b1c8fe5710b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } health_report { overall_health: UNKNOWN } } }
I20260923 20:46:22.014765 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.009464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.5.71.130:38519)
0923 20:46:22.010051 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:22.010059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:22.010082 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:46:22.010151 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:22.010154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:22.010214 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:22.010317 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:22.010761 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.011591 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.012972 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.013401 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.013536 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.013967 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:22.014071 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:22.014178 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:22.014335 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 20:46:22.014411 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":309,"threads_started":1}
I20260923 20:46:22.024592 7606 raft_consensus.cc:1261] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Refusing update from remote peer 4d59002757c647cc95560b1c8fe5710b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:22.024719 7321 raft_consensus.cc:1261] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Refusing update from remote peer 4d59002757c647cc95560b1c8fe5710b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:22.025301 7774 consensus_queue.cc:1035] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [LEADER]: Connected to new peer: Peer: permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:22.025636 7773 consensus_queue.cc:1035] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:22.069839 7164 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:22.070068 7164 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:22.073140 7164 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Sending DeleteTablet for 3 replicas of tablet 5b748ef7cf9249bdb023e384c6f17144
I20260923 20:46:22.074147 7302 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b748ef7cf9249bdb023e384c6f17144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:22 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57068
I20260923 20:46:22.074225 7580 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b748ef7cf9249bdb023e384c6f17144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:22 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:39784
I20260923 20:46:22.074241 7440 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b748ef7cf9249bdb023e384c6f17144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:22 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:34318
I20260923 20:46:22.074545 7756 tablet_replica.cc:330] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: stopping tablet replica
I20260923 20:46:22.074548 7757 tablet_replica.cc:330] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: stopping tablet replica
I20260923 20:46:22.074816 7756 raft_consensus.cc:2229] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:22.074812 7757 raft_consensus.cc:2229] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:22.075085 7756 raft_consensus.cc:2258] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:22.075176 7757 raft_consensus.cc:2258] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:22.075212 7755 tablet_replica.cc:330] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: stopping tablet replica
I20260923 20:46:22.075419 7755 raft_consensus.cc:2229] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:22.075641 7755 raft_consensus.cc:2258] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:22.075999 7164 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 20:46:22.076359 7757 ts_tablet_manager.cc:1905] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:22.076424 7756 ts_tablet_manager.cc:1905] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 20:46:22.076805 7164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:22.078884 7755 ts_tablet_manager.cc:1905] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:22.082485 7757 ts_tablet_manager.cc:1918] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:22.082671 7757 log.cc:1196] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/5b748ef7cf9249bdb023e384c6f17144
I20260923 20:46:22.083104 7757 ts_tablet_manager.cc:1939] T 5b748ef7cf9249bdb023e384c6f17144 P 4d59002757c647cc95560b1c8fe5710b: Deleting consensus metadata
I20260923 20:46:22.083971 7756 ts_tablet_manager.cc:1918] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:22.084165 7756 log.cc:1196] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/5b748ef7cf9249bdb023e384c6f17144
I20260923 20:46:22.084627 7756 ts_tablet_manager.cc:1939] T 5b748ef7cf9249bdb023e384c6f17144 P 329d0db9e44c4661989fed81051123fa: Deleting consensus metadata
I20260923 20:46:22.084761 7152 catalog_manager.cc:4841] TS 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519): tablet 5b748ef7cf9249bdb023e384c6f17144 (table test-table [id=71ef76737caf4354808b53a5bb7305dc]) successfully deleted
I20260923 20:46:22.086007 7302 tablet_service.cc:1467] Processing CreateTablet for tablet b625baaec0f044e3b6d69b26890fbc5a (DEFAULT_TABLE table=test-table [id=41ea2a5a2b16433ca2e725b6768674bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:22.086335 7440 tablet_service.cc:1467] Processing CreateTablet for tablet b625baaec0f044e3b6d69b26890fbc5a (DEFAULT_TABLE table=test-table [id=41ea2a5a2b16433ca2e725b6768674bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:22.086395 7151 catalog_manager.cc:4841] TS 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379): tablet 5b748ef7cf9249bdb023e384c6f17144 (table test-table [id=71ef76737caf4354808b53a5bb7305dc]) successfully deleted
I20260923 20:46:22.086510 7302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b625baaec0f044e3b6d69b26890fbc5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:22.086723 7440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b625baaec0f044e3b6d69b26890fbc5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:22.088762 7580 tablet_service.cc:1467] Processing CreateTablet for tablet b625baaec0f044e3b6d69b26890fbc5a (DEFAULT_TABLE table=test-table [id=41ea2a5a2b16433ca2e725b6768674bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:22.089509 7580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b625baaec0f044e3b6d69b26890fbc5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:22.092125 7755 ts_tablet_manager.cc:1918] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:46:22.092308 7755 log.cc:1196] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/5b748ef7cf9249bdb023e384c6f17144
I20260923 20:46:22.093238 7768 tablet_bootstrap.cc:492] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: Bootstrap starting.
I20260923 20:46:22.094034 7755 ts_tablet_manager.cc:1939] T 5b748ef7cf9249bdb023e384c6f17144 P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting consensus metadata
I20260923 20:46:22.095134 7768 tablet_bootstrap.cc:654] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:22.095245 7151 catalog_manager.cc:4841] TS 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049): tablet 5b748ef7cf9249bdb023e384c6f17144 (table test-table [id=71ef76737caf4354808b53a5bb7305dc]) successfully deleted
I20260923 20:46:22.095564 7767 tablet_bootstrap.cc:492] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Bootstrap starting.
I20260923 20:46:22.097196 7767 tablet_bootstrap.cc:654] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:22.097642 7768 tablet_bootstrap.cc:492] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: No bootstrap required, opened a new log
I20260923 20:46:22.097822 7768 ts_tablet_manager.cc:1397] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260923 20:46:22.098642 7768 raft_consensus.cc:348] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.098929 7768 raft_consensus.cc:374] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:22.099010 7768 raft_consensus.cc:729] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59002757c647cc95560b1c8fe5710b, State: Initialized, Role: FOLLOWER
I20260923 20:46:22.099341 7767 tablet_bootstrap.cc:492] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: No bootstrap required, opened a new log
I20260923 20:46:22.099269 7768 consensus_queue.cc:260] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.099531 7767 ts_tablet_manager.cc:1397] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260923 20:46:22.099936 7768 ts_tablet_manager.cc:1428] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260923 20:46:22.100348 7767 raft_consensus.cc:348] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.100584 7767 raft_consensus.cc:374] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:22.100666 7767 raft_consensus.cc:729] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 329d0db9e44c4661989fed81051123fa, State: Initialized, Role: FOLLOWER
I20260923 20:46:22.100926 7767 consensus_queue.cc:260] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.101084 7766 tablet_bootstrap.cc:492] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: Bootstrap starting.
I20260923 20:46:22.101573 7767 ts_tablet_manager.cc:1428] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 20:46:22.102659 7766 tablet_bootstrap.cc:654] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:22.104777 7766 tablet_bootstrap.cc:492] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: No bootstrap required, opened a new log
I20260923 20:46:22.105018 7766 ts_tablet_manager.cc:1397] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260923 20:46:22.105895 7766 raft_consensus.cc:348] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.106165 7766 raft_consensus.cc:374] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:22.106263 7766 raft_consensus.cc:729] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5addcbb4bcd24b5aafd90f7d737a2f49, State: Initialized, Role: FOLLOWER
I20260923 20:46:22.106529 7766 consensus_queue.cc:260] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.107110 7766 ts_tablet_manager.cc:1428] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260923 20:46:22.156934 7772 raft_consensus.cc:482] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:22.157204 7772 raft_consensus.cc:504] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.158924 7772 leader_election.cc:288] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519), 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379)
I20260923 20:46:22.167328 7777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.159393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40647 (local address 127.5.71.130:38519)
0923 20:46:22.159582 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:22.159589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:22.160827 (+ 1238us) server_negotiation.cc:407] Connection header received
0923 20:46:22.161054 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:22.161060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:22.161131 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:22.161230 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:22.161851 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.162537 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.164970 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.165902 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.166087 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.166213 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:22.166708 (+ 495us) server_negotiation.cc:299] Negotiation successful
0923 20:46:22.166900 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 20:46:22.167444 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.159283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.5.71.129:56097)
0923 20:46:22.160403 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:22.160423 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:22.160602 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:22.161008 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:22.161013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:22.161038 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:22.161433 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.161443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.162616 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.162623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.164517 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.164533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.166384 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.166391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.166546 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.166577 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 20:46:22.166664 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 20:46:22.166773 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":1,"thread_start_us":1083,"threads_started":1}
I20260923 20:46:22.167703 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.159512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:56097 (local address 127.5.71.131:41379)
0923 20:46:22.160138 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:22.160148 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:22.160517 (+ 369us) server_negotiation.cc:407] Connection header received
0923 20:46:22.160811 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:22.160818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:22.160888 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:22.160992 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:22.161596 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.162476 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.164667 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.165939 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.166192 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.166247 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:22.166844 (+ 597us) server_negotiation.cc:299] Negotiation successful
0923 20:46:22.167027 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":297,"threads_started":1}
I20260923 20:46:22.168090 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.159292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.5.71.129:40647)
0923 20:46:22.160702 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:22.160727 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:22.160886 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:22.161277 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:22.161282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:22.161308 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:22.161638 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.161674 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.162677 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.162688 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.164806 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.164818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.166206 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.166212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.166492 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.166529 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:46:22.166633 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 20:46:22.166771 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":53,"thread_start_us":719,"threads_started":1}
I20260923 20:46:22.168787 7606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b625baaec0f044e3b6d69b26890fbc5a" candidate_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329d0db9e44c4661989fed81051123fa" is_pre_election: true
I20260923 20:46:22.169025 7465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b625baaec0f044e3b6d69b26890fbc5a" candidate_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b" is_pre_election: true
I20260923 20:46:22.169173 7606 raft_consensus.cc:2454] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5addcbb4bcd24b5aafd90f7d737a2f49 in term 0.
I20260923 20:46:22.169405 7465 raft_consensus.cc:2454] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5addcbb4bcd24b5aafd90f7d737a2f49 in term 0.
I20260923 20:46:22.169845 7254 leader_election.cc:302] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:22.170502 7772 raft_consensus.cc:2781] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:22.170800 7772 raft_consensus.cc:482] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:22.170974 7772 raft_consensus.cc:3037] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:22.174190 7772 raft_consensus.cc:504] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.175062 7772 leader_election.cc:288] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [CANDIDATE]: Term 1 election: Requested vote from peers 4d59002757c647cc95560b1c8fe5710b (127.5.71.130:38519), 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379)
I20260923 20:46:22.175575 7465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b625baaec0f044e3b6d69b26890fbc5a" candidate_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d59002757c647cc95560b1c8fe5710b"
I20260923 20:46:22.175990 7465 raft_consensus.cc:3037] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:22.175973 7606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b625baaec0f044e3b6d69b26890fbc5a" candidate_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "329d0db9e44c4661989fed81051123fa"
I20260923 20:46:22.176271 7606 raft_consensus.cc:3037] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:22.180851 7606 raft_consensus.cc:2454] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5addcbb4bcd24b5aafd90f7d737a2f49 in term 1.
I20260923 20:46:22.180851 7465 raft_consensus.cc:2454] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5addcbb4bcd24b5aafd90f7d737a2f49 in term 1.
I20260923 20:46:22.181461 7254 leader_election.cc:302] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 329d0db9e44c4661989fed81051123fa, 5addcbb4bcd24b5aafd90f7d737a2f49; no voters:
I20260923 20:46:22.182036 7772 raft_consensus.cc:2781] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:22.182956 7772 raft_consensus.cc:686] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 LEADER]: Becoming Leader. State: Replica: 5addcbb4bcd24b5aafd90f7d737a2f49, State: Running, Role: LEADER
I20260923 20:46:22.183475 7772 consensus_queue.cc:237] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } }
I20260923 20:46:22.188213 7163 catalog_manager.cc:5495] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addcbb4bcd24b5aafd90f7d737a2f49" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 } health_report { overall_health: UNKNOWN } } }
I20260923 20:46:22.229199 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.220169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57070 (local address 127.5.71.129:39049)
0923 20:46:22.221345 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:22.221350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:22.221382 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:46:22.221468 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:22.221471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:22.221564 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:22.221719 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:22.222476 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.223693 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.226015 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.226605 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.226706 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.227545 (+ 839us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:22.228346 (+ 801us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:22.228462 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:22.228664 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 20:46:22.228735 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":785,"threads_started":1}
I20260923 20:46:22.248275 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.242875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:49159 (local address 127.5.71.131:41379)
0923 20:46:22.243051 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:22.243055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:22.243634 (+ 579us) server_negotiation.cc:407] Connection header received
0923 20:46:22.243771 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:22.243775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:22.243911 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:22.243995 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:22.244546 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.245290 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.246667 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.247092 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.247212 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.247610 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:22.247729 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:22.247887 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:22.247984 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 20:46:22.248047 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 20:46:22.248332 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.242485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41379 (local address 127.5.71.129:49159)
0923 20:46:22.243514 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:22.243530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:22.243656 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:22.244025 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:22.244028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:22.244038 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:22.244408 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.244418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.245410 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.245413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.246514 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.246525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.247266 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.247271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.247398 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.247453 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:22.247878 (+ 425us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:22.247887 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 20:46:22.247951 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 20:46:22.248056 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"thread_start_us":712,"threads_started":1}
I20260923 20:46:22.258112 7465 raft_consensus.cc:1261] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Refusing update from remote peer 5addcbb4bcd24b5aafd90f7d737a2f49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:22.258154 7606 raft_consensus.cc:1261] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Refusing update from remote peer 5addcbb4bcd24b5aafd90f7d737a2f49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:22.258922 7788 consensus_queue.cc:1035] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d59002757c647cc95560b1c8fe5710b" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:22.259392 7772 consensus_queue.cc:1035] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "329d0db9e44c4661989fed81051123fa" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:22.295020 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.286820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:53075 (local address 127.5.71.129:39049)
0923 20:46:22.287024 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:22.287028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:22.289967 (+ 2939us) server_negotiation.cc:407] Connection header received
0923 20:46:22.290154 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:22.290161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:22.290275 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:22.290385 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:22.290951 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.291888 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.293232 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.293650 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.293799 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.294206 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:22.294322 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:22.294420 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:22.294619 (+ 199us) server_negotiation.cc:299] Negotiation successful
0923 20:46:22.294698 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:46:22.295080 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:22.286704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.5.71.131:53075)
0923 20:46:22.289835 (+ 3131us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:22.289851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:22.290015 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:22.290405 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:22.290411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:22.290424 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:46:22.290797 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.290805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.292051 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.292057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.293083 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:22.293094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:22.293850 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:22.293859 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:22.294000 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:22.294037 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:46:22.294428 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:46:22.294443 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 20:46:22.294591 (+ 148us) client_negotiation.cc:240] Negotiation successful
0923 20:46:22.294731 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2973,"thread_start_us":2842,"threads_started":1}
W20260923 20:46:22.296279 7739 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 20:46:23.898166 7163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:23.898442 7163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:23.903649 7163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a33c53d75274df29c88584a6d1e421c: Sending DeleteTablet for 3 replicas of tablet b625baaec0f044e3b6d69b26890fbc5a
I20260923 20:46:23.904608 7302 tablet_service.cc:1514] Processing DeleteTablet for tablet b625baaec0f044e3b6d69b26890fbc5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:23 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57068
I20260923 20:46:23.904682 7440 tablet_service.cc:1514] Processing DeleteTablet for tablet b625baaec0f044e3b6d69b26890fbc5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:23 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:34318
I20260923 20:46:23.904965 7580 tablet_service.cc:1514] Processing DeleteTablet for tablet b625baaec0f044e3b6d69b26890fbc5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:23 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:39784
I20260923 20:46:23.906431 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7236
I20260923 20:46:23.919643 7823 tablet_replica.cc:330] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: stopping tablet replica
I20260923 20:46:23.919889 7823 raft_consensus.cc:2229] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:23.920133 7823 raft_consensus.cc:2258] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:23.920856 7822 tablet_replica.cc:330] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: stopping tablet replica
I20260923 20:46:23.921051 7822 raft_consensus.cc:2229] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:23.921077 7824 tablet_replica.cc:330] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: stopping tablet replica
I20260923 20:46:23.921399 7824 raft_consensus.cc:2229] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:23.921613 7822 raft_consensus.cc:2258] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:23.921756 7824 raft_consensus.cc:2258] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:23.922581 7823 ts_tablet_manager.cc:1905] T b625baaec0f044e3b6d69b26890fbc5a P 4d59002757c647cc95560b1c8fe5710b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:23.923544 7822 ts_tablet_manager.cc:1905] T b625baaec0f044e3b6d69b26890fbc5a P 5addcbb4bcd24b5aafd90f7d737a2f49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:23.924494 7824 ts_tablet_manager.cc:1905] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 20:46:23 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:23.942773 7826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:23.908661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57074 (local address 127.5.71.129:39049)
0923 20:46:23.915154 (+ 6493us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:23.915164 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:23.915211 (+ 47us) server_negotiation.cc:407] Connection header received
0923 20:46:23.915299 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:23.915308 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:23.915413 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:23.919392 (+ 3979us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:23.919678 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:23.921058 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:23.922818 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:23.923533 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:23.924366 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:23.935960 (+ 11594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:23.935978 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:23.935998 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:23.936039 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:23.939058 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:23.940139 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:23.940150 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:23.940154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:23.940335 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:23.940888 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:23.940893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:23.940895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:23.941277 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:23.941516 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:23.942013 (+ 497us) server_negotiation.cc:299] Negotiation successful
0923 20:46:23.942215 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6311,"thread_start_us":6090,"threads_started":1}
W20260923 20:46:23.943657 7367 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57074
W20260923 20:46:23.944608 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7236: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:23.944844 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 7236
I20260923 20:46:24.001336 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7376
W20260923 20:46:24.001407 7151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:39049 (error 108)
I20260923 20:46:24.004168 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.002918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.0.0.1:57078)
0923 20:46:24.003691 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.003925 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:39049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":525,"threads_started":1}
W20260923 20:46:24.004431 7151 catalog_manager.cc:4584] TS 5addcbb4bcd24b5aafd90f7d737a2f49 (127.5.71.129:39049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b625baaec0f044e3b6d69b26890fbc5a: Network error: Client connection negotiation failed: client connection to 127.5.71.129:39049: connect: Connection refused (error 111)
Sep 23 20:46:24 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:24.019577 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.002221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.5.71.130:38519)
0923 20:46:24.002792 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:24.002806 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:24.002827 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:46:24.002893 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:24.002898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:24.002974 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:24.003109 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:24.004592 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:24.005357 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:24.006947 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:24.007394 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:24.007513 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:24.012601 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:24.012653 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:24.012679 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:24.012730 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:24.016363 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:24.017141 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:24.017150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:24.017152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:24.017379 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:24.017891 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:24.017895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:24.017897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:24.018259 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:24.018504 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:24.018990 (+ 486us) server_negotiation.cc:299] Negotiation successful
0923 20:46:24.019234 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":304,"threads_started":1}
W20260923 20:46:24.020277 7506 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34354
W20260923 20:46:24.021126 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7376: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:24.021292 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 7376
I20260923 20:46:24.058228 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.057551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.0.0.1:57086)
0923 20:46:24.057852 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.057971 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:39049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260923 20:46:24.059404 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.058994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.0.0.1:57096)
0923 20:46:24.059165 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.059253 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:39049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260923 20:46:24.076742 7824 ts_tablet_manager.cc:1918] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 20:46:24.076965 7824 log.cc:1196] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/b625baaec0f044e3b6d69b26890fbc5a
I20260923 20:46:24.077406 7824 ts_tablet_manager.cc:1939] T b625baaec0f044e3b6d69b26890fbc5a P 329d0db9e44c4661989fed81051123fa: Deleting consensus metadata
I20260923 20:46:24.078601 7151 catalog_manager.cc:4841] TS 329d0db9e44c4661989fed81051123fa (127.5.71.131:41379): tablet b625baaec0f044e3b6d69b26890fbc5a (table test-table [id=41ea2a5a2b16433ca2e725b6768674bc]) successfully deleted
I20260923 20:46:24.100373 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7515
I20260923 20:46:24.100744 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.100267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.0.0.1:34360)
0923 20:46:24.100464 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.100559 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.130:38519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
Sep 23 20:46:24 dist-test-slave-dn29 krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196381, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:24.116632 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.101402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.5.71.131:41379)
0923 20:46:24.101917 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:24.101922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:24.101938 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:46:24.101992 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:24.101996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:24.102054 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:24.102150 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:24.103621 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:24.104502 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:24.105875 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:24.106280 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:24.106390 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:24.110783 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:24.110852 (+ 69us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:24.110871 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:24.110916 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:24.114009 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:24.114637 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:24.114649 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:24.114656 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:24.114838 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:24.115205 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:24.115208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:24.115210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:24.115547 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:24.115767 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:24.116091 (+ 324us) server_negotiation.cc:299] Negotiation successful
0923 20:46:24.116252 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":262,"threads_started":1}
W20260923 20:46:24.117368 7646 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39816
W20260923 20:46:24.117998 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7515: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:24.118216 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 7515
I20260923 20:46:24.120751 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.120301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.0.0.1:57108)
0923 20:46:24.120512 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.120594 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:39049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260923 20:46:24.121857 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.121482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39049 (local address 127.0.0.1:57122)
0923 20:46:24.121669 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.121732 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:39049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260923 20:46:24.133351 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.132880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.0.0.1:34366)
0923 20:46:24.133095 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.133183 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.130:38519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260923 20:46:24.134615 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.134206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.0.0.1:34376)
0923 20:46:24.134424 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.134499 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.130:38519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260923 20:46:24.144618 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7133
I20260923 20:46:24.145942 7205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:24.270582 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:24.269559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38519 (local address 127.0.0.1:34380)
0923 20:46:24.270240 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:24.270344 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.130:38519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458}
I20260923 20:46:24.271212 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 7133
2026-09-23T20:46:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9119 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:24 dist-test-slave-dn29 krb5kdc[7835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:24 dist-test-slave-dn29 krb5kdc[7835](info): set up 2 sockets
Sep 23 20:46:24 dist-test-slave-dn29 krb5kdc[7835](info): commencing operation
krb5kdc: starting...
W20260923 20:46:25.564584 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.009s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:25 dist-test-slave-dn29 krb5kdc[7835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:25.787719 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:32783
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:32783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:26.137851 7848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:26.138157 7848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:26.138314 7848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:26.145764 7848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:26.145885 7848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:26.145982 7848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:26.146095 7848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:26.146159 7848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:26.157070 7848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:32783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:32783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:26.158464 7848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:26.159852 7848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:26.172608 7853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:26.172569 7854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:26.172618 7856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:26.173285 7848 server_base.cc:1034] running on GCE node
I20260923 20:46:26.174161 7848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:26.175637 7848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:26.176893 7848 hybrid_clock.cc:648] HybridClock initialized: now 1790196386176871 us; error 79 us; skew 500 ppm
I20260923 20:46:26.177328 7848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:26 dist-test-slave-dn29 krb5kdc[7835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:26.181051 7848 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:26.182770 7848 webserver.cc:460] Webserver started at http://127.5.71.190:34883/ using document root <none> and password file <none>
I20260923 20:46:26.183595 7848 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:26.183754 7848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:26.184172 7848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:26.187543 7848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "52b15ee9f28d4ccd992ed650eb8596e1"
format_stamp: "Formatted at 2026-09-23 20:46:26 on dist-test-slave-dn29"
server_key: "4687dc55b9ebbb6d663aad763b81df93"
server_key_iv: "bbbdfa90783f9e4b60b4d7a76ee7c296"
server_key_version: "encryptionkey@0"
I20260923 20:46:26.188552 7848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "52b15ee9f28d4ccd992ed650eb8596e1"
format_stamp: "Formatted at 2026-09-23 20:46:26 on dist-test-slave-dn29"
server_key: "4687dc55b9ebbb6d663aad763b81df93"
server_key_iv: "bbbdfa90783f9e4b60b4d7a76ee7c296"
server_key_version: "encryptionkey@0"
I20260923 20:46:26.194602 7848 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:46:26.198704 7863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:26.200425 7848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:26.200619 7848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "52b15ee9f28d4ccd992ed650eb8596e1"
format_stamp: "Formatted at 2026-09-23 20:46:26 on dist-test-slave-dn29"
server_key: "4687dc55b9ebbb6d663aad763b81df93"
server_key_iv: "bbbdfa90783f9e4b60b4d7a76ee7c296"
server_key_version: "encryptionkey@0"
I20260923 20:46:26.200822 7848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:26.219760 7848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:26.224169 7848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:26.224475 7848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:26.244117 7848 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:32783
I20260923 20:46:26.244140 7914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:32783 every 8 connection(s)
I20260923 20:46:26.245609 7848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:26.248209 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 7848
I20260923 20:46:26.248456 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:26.249095 5406 external_mini_cluster.cc:1428] Setting key 6cadf67f93c191474c10875c11abf5b9
I20260923 20:46:26.249753 7915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:26.263947 7915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1: Bootstrap starting.
Sep 23 20:46:26 dist-test-slave-dn29 krb5kdc[7835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:26.267592 7915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:26.268759 7915 log.cc:824] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:26.271847 7915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1: No bootstrap required, opened a new log
I20260923 20:46:26.273370 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:26.251171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51658 (local address 127.5.71.190:32783)
0923 20:46:26.251996 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:26.252011 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:26.252058 (+ 47us) server_negotiation.cc:407] Connection header received
0923 20:46:26.254543 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:26.254573 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:26.255143 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:26.255594 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:26.257021 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:26.258957 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:26.260367 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:26.261476 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:26.261672 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:26.266002 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:26.266011 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:26.266027 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:26.266070 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:26.269238 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:26.269962 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:26.269969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:26.269976 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:26.270141 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:26.270531 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:26.270540 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:26.270542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:26.271152 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:26.271477 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:26.271932 (+ 455us) server_negotiation.cc:299] Negotiation successful
0923 20:46:26.272379 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":344,"threads_started":1}
I20260923 20:46:26.280248 7915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 32783 } }
I20260923 20:46:26.280663 7915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:26.280771 7915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b15ee9f28d4ccd992ed650eb8596e1, State: Initialized, Role: FOLLOWER
I20260923 20:46:26.281334 7915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 32783 } }
I20260923 20:46:26.281575 7915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:26.281721 7915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:26.281898 7915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:26.284591 7915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 32783 } }
I20260923 20:46:26.285112 7915 leader_election.cc:302] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b15ee9f28d4ccd992ed650eb8596e1; no voters:
I20260923 20:46:26.285727 7915 leader_election.cc:288] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:26.285887 7920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:26.286713 7920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [term 1 LEADER]: Becoming Leader. State: Replica: 52b15ee9f28d4ccd992ed650eb8596e1, State: Running, Role: LEADER
I20260923 20:46:26.287242 7920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 32783 } }
I20260923 20:46:26.288008 7915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:26.290474 7922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b15ee9f28d4ccd992ed650eb8596e1. Latest consensus state: current_term: 1 leader_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 32783 } } }
I20260923 20:46:26.290521 7921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b15ee9f28d4ccd992ed650eb8596e1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 32783 } } }
I20260923 20:46:26.290953 7922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:26.291044 7921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:26.292735 7928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:26.298237 7928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:26.308561 7928 catalog_manager.cc:1347] Generated new cluster ID: 96b5ff3d784e40b19f1989cb0556fe55
I20260923 20:46:26.308709 7928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:26.323520 7928 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:26.324996 7928 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:26.333580 7928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52b15ee9f28d4ccd992ed650eb8596e1: Generated new TSK 0
I20260923 20:46:26.334801 7928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:26.410845 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:32783
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:26.757273 7941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:26.757653 7941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:26.757853 7941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:26.765588 7941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:26.765769 7941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:26.765951 7941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:26.776417 7941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:32783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:26.777837 7941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:26.779191 7941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:26.793134 7949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:26.793191 7947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:26.793133 7946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:26.794193 7941 server_base.cc:1034] running on GCE node
I20260923 20:46:26.794832 7941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:26.795940 7941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:26.797200 7941 hybrid_clock.cc:648] HybridClock initialized: now 1790196386797147 us; error 43 us; skew 500 ppm
I20260923 20:46:26.797569 7941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:26 dist-test-slave-dn29 krb5kdc[7835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:26.801303 7941 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:26.803149 7941 webserver.cc:460] Webserver started at http://127.5.71.129:41899/ using document root <none> and password file <none>
I20260923 20:46:26.803966 7941 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:26.804078 7941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:26.804347 7941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:26.807304 7941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "7c351ea46b3641b9b194fbacf2cf0df4"
format_stamp: "Formatted at 2026-09-23 20:46:26 on dist-test-slave-dn29"
server_key: "1d4321829ddac00e44f721167b22b3e7"
server_key_iv: "6d14ed0bf30a8489783fbdcefa6dddc0"
server_key_version: "encryptionkey@0"
I20260923 20:46:26.808291 7941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "7c351ea46b3641b9b194fbacf2cf0df4"
format_stamp: "Formatted at 2026-09-23 20:46:26 on dist-test-slave-dn29"
server_key: "1d4321829ddac00e44f721167b22b3e7"
server_key_iv: "6d14ed0bf30a8489783fbdcefa6dddc0"
server_key_version: "encryptionkey@0"
I20260923 20:46:26.813712 7941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:26.817351 7956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:26.818866 7941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:26.819139 7941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "7c351ea46b3641b9b194fbacf2cf0df4"
format_stamp: "Formatted at 2026-09-23 20:46:26 on dist-test-slave-dn29"
server_key: "1d4321829ddac00e44f721167b22b3e7"
server_key_iv: "6d14ed0bf30a8489783fbdcefa6dddc0"
server_key_version: "encryptionkey@0"
I20260923 20:46:26.819348 7941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:26.853387 7941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:26.857731 7941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:26.858031 7941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:26.859061 7941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:26.860495 7941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:26.860569 7941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:26.860699 7941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:26.860759 7941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:26.896376 7941 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:39591
I20260923 20:46:26.896533 8068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:39591 every 8 connection(s)
I20260923 20:46:26.897866 7941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:26.903347 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 7941
I20260923 20:46:26.903564 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:26.904328 5406 external_mini_cluster.cc:1428] Setting key 37690ba8b7f0ea246edd0b3c510899cd
Sep 23 20:46:26 dist-test-slave-dn29 krb5kdc[7835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196386, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:26.921278 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:26.900624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:39381 (local address 127.5.71.190:32783)
0923 20:46:26.901056 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:26.901061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:26.902075 (+ 1014us) server_negotiation.cc:407] Connection header received
0923 20:46:26.904126 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:26.904132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:26.904229 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:26.904345 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:26.906180 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:26.906887 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:26.908577 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:26.908969 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:26.909103 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:26.913446 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:26.913458 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:26.913467 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:26.913518 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:26.917442 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:26.918274 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:26.918283 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:26.918290 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:26.918452 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:26.918915 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:26.918924 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:26.918931 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:26.919361 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:26.919537 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:26.920421 (+ 884us) server_negotiation.cc:299] Negotiation successful
0923 20:46:26.920719 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":222,"threads_started":1}
I20260923 20:46:26.921893 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:26.901237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:32783 (local address 127.5.71.129:39381)
0923 20:46:26.901908 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:26.901948 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:26.903781 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:26.904604 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:26.904613 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:26.905309 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:26.906009 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:26.906034 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:26.907047 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:26.907053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:26.908415 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:26.908427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:26.909221 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:26.909229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:26.909457 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:26.910230 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:26.910261 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:26.913191 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:26.917636 (+ 4445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:26.917644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:26.917661 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:26.918109 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:26.918580 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:26.918583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:26.918585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:26.918776 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:26.919520 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:26.919529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:26.919996 (+ 467us) client_negotiation.cc:769] Sending connection context
0923 20:46:26.920234 (+ 238us) client_negotiation.cc:240] Negotiation successful
0923 20:46:26.920659 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":287,"threads_started":1}
I20260923 20:46:26.923753 8069 heartbeater.cc:347] Connected to a master server at 127.5.71.190:32783
I20260923 20:46:26.924152 8069 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:26.925050 8069 heartbeater.cc:511] Master 127.5.71.190:32783 requested a full tablet report, sending...
I20260923 20:46:26.927388 7880 ts_manager.cc:194] Registered new tserver with Master: 7c351ea46b3641b9b194fbacf2cf0df4 (127.5.71.129:39591)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:26.929253 7880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:39381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:26.952137 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:32783
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:27.289719 8075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:27.290042 8075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:27.290138 8075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:27.297503 8075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:27.297624 8075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:27.297739 8075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:27.307974 8075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:32783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:27.309274 8075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:27.310700 8075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:27.324491 8083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:27.324520 8081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:27.324520 8080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:27.325218 8075 server_base.cc:1034] running on GCE node
I20260923 20:46:27.325798 8075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:27.326716 8075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:27.327912 8075 hybrid_clock.cc:648] HybridClock initialized: now 1790196387327807 us; error 97 us; skew 500 ppm
I20260923 20:46:27.328266 8075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:27 dist-test-slave-dn29 krb5kdc[7835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:27.331468 8075 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:27.333094 8075 webserver.cc:460] Webserver started at http://127.5.71.130:39829/ using document root <none> and password file <none>
I20260923 20:46:27.333818 8075 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:27.333942 8075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:27.334265 8075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:27.337231 8075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "cc13d567233d4a6aa7123bd2ab92f213"
format_stamp: "Formatted at 2026-09-23 20:46:27 on dist-test-slave-dn29"
server_key: "e7cc1c4fa845aa8870e7aaada024de6e"
server_key_iv: "b907587f00fff69ff23ad5344c0d823d"
server_key_version: "encryptionkey@0"
I20260923 20:46:27.338032 8075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "cc13d567233d4a6aa7123bd2ab92f213"
format_stamp: "Formatted at 2026-09-23 20:46:27 on dist-test-slave-dn29"
server_key: "e7cc1c4fa845aa8870e7aaada024de6e"
server_key_iv: "b907587f00fff69ff23ad5344c0d823d"
server_key_version: "encryptionkey@0"
I20260923 20:46:27.343339 8075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260923 20:46:27.346975 8090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:27.348474 8075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260923 20:46:27.348660 8075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "cc13d567233d4a6aa7123bd2ab92f213"
format_stamp: "Formatted at 2026-09-23 20:46:27 on dist-test-slave-dn29"
server_key: "e7cc1c4fa845aa8870e7aaada024de6e"
server_key_iv: "b907587f00fff69ff23ad5344c0d823d"
server_key_version: "encryptionkey@0"
I20260923 20:46:27.348887 8075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:27.363032 8075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:27.367614 8075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:27.368028 8075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:27.369130 8075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:27.370666 8075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:27.370782 8075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:27.370940 8075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:27.371008 8075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:27.409366 8075 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38835
I20260923 20:46:27.409431 8202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38835 every 8 connection(s)
I20260923 20:46:27.410852 8075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:27.412096 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8075
I20260923 20:46:27.412317 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:27.412782 5406 external_mini_cluster.cc:1428] Setting key cde63665826f80a25acd80878a0ef444
Sep 23 20:46:27 dist-test-slave-dn29 krb5kdc[7835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:27.435079 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:27.413889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33689 (local address 127.5.71.190:32783)
0923 20:46:27.414097 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:27.414102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:27.415749 (+ 1647us) server_negotiation.cc:407] Connection header received
0923 20:46:27.417831 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:27.417851 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:27.417955 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:27.418078 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:27.419785 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.420990 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.423003 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.423927 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.424107 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:27.428380 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:27.428386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:27.428389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:27.428464 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:27.431232 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:27.432416 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:27.432420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:27.432422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:27.432554 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:27.433159 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:27.433163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:27.433165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:27.433481 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:27.433601 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:27.434541 (+ 940us) server_negotiation.cc:299] Negotiation successful
0923 20:46:27.434769 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:27.436455 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:27.414589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:32783 (local address 127.5.71.130:33689)
0923 20:46:27.415539 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:27.415598 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:27.417491 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:27.418397 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:27.418407 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:27.418961 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:27.419591 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:27.419610 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.421186 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.421191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:27.422818 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:27.422835 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.424264 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.424271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:27.424522 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:27.425296 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:27.425320 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:27.428072 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:27.431430 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:27.431442 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:27.431463 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:27.432274 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:27.432703 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:27.432711 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:27.432718 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:27.433023 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:27.433619 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:27.433626 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:27.434141 (+ 515us) client_negotiation.cc:769] Sending connection context
0923 20:46:27.434404 (+ 263us) client_negotiation.cc:240] Negotiation successful
0923 20:46:27.434867 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":408,"threads_started":1}
I20260923 20:46:27.438453 8203 heartbeater.cc:347] Connected to a master server at 127.5.71.190:32783
I20260923 20:46:27.438925 8203 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:27.439736 8203 heartbeater.cc:511] Master 127.5.71.190:32783 requested a full tablet report, sending...
I20260923 20:46:27.441493 7880 ts_manager.cc:194] Registered new tserver with Master: cc13d567233d4a6aa7123bd2ab92f213 (127.5.71.130:38835)
I20260923 20:46:27.442905 7880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:33689
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:27.465358 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:32783
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:27.805269 8208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:27.805565 8208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:27.805661 8208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:27.813167 8208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:27.813297 8208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:27.813410 8208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:27.823606 8208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:32783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:27.825053 8208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:27.826457 8208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:27.840062 8213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:27.840026 8214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:27.840528 8216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:27.841456 8208 server_base.cc:1034] running on GCE node
I20260923 20:46:27.842096 8208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:27.843014 8208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:27.844216 8208 hybrid_clock.cc:648] HybridClock initialized: now 1790196387844113 us; error 82 us; skew 500 ppm
I20260923 20:46:27.844553 8208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:27 dist-test-slave-dn29 krb5kdc[7835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:27.847752 8208 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:27.849360 8208 webserver.cc:460] Webserver started at http://127.5.71.131:42633/ using document root <none> and password file <none>
I20260923 20:46:27.850075 8208 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:27.850224 8208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:27.850538 8208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:27.853895 8208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "52c02eabb48b442abe8739b9fc1ddbed"
format_stamp: "Formatted at 2026-09-23 20:46:27 on dist-test-slave-dn29"
server_key: "670febf2ab33832b6e79908ad011df39"
server_key_iv: "e17a3e7cd6fd18885c832d3d1ca2cee8"
server_key_version: "encryptionkey@0"
I20260923 20:46:27.854725 8208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "52c02eabb48b442abe8739b9fc1ddbed"
format_stamp: "Formatted at 2026-09-23 20:46:27 on dist-test-slave-dn29"
server_key: "670febf2ab33832b6e79908ad011df39"
server_key_iv: "e17a3e7cd6fd18885c832d3d1ca2cee8"
server_key_version: "encryptionkey@0"
I20260923 20:46:27.860540 8208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260923 20:46:27.864790 8223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:27.866284 8208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260923 20:46:27.866485 8208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "52c02eabb48b442abe8739b9fc1ddbed"
format_stamp: "Formatted at 2026-09-23 20:46:27 on dist-test-slave-dn29"
server_key: "670febf2ab33832b6e79908ad011df39"
server_key_iv: "e17a3e7cd6fd18885c832d3d1ca2cee8"
server_key_version: "encryptionkey@0"
I20260923 20:46:27.866710 8208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:27.893275 8208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:27.898743 8208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:27.899058 8208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:27.900372 8208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:27.902052 8208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:27.902144 8208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:27.902264 8208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:27.902328 8208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:27.932806 8069 heartbeater.cc:503] Master 127.5.71.190:32783 was elected leader, sending a full tablet report...
I20260923 20:46:27.938475 8208 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:39675
I20260923 20:46:27.938519 8335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:39675 every 8 connection(s)
I20260923 20:46:27.939844 8208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:27.945906 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8208
I20260923 20:46:27.946151 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:27.946733 5406 external_mini_cluster.cc:1428] Setting key 4d25c1d88119a9014453baa0fa3bf513
Sep 23 20:46:27 dist-test-slave-dn29 krb5kdc[7835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196387, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:27.962430 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:27.942472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:34675 (local address 127.5.71.190:32783)
0923 20:46:27.942947 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:27.942951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:27.943960 (+ 1009us) server_negotiation.cc:407] Connection header received
0923 20:46:27.945859 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:27.945865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:27.945964 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:27.946098 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:27.948278 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.949304 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.951218 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.951874 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.952016 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:27.956124 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:27.956129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:27.956132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:27.956167 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:27.958799 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:27.959497 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:27.959501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:27.959503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:27.959619 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:27.960349 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:27.960355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:27.960361 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:27.960711 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:27.960901 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:27.961858 (+ 957us) server_negotiation.cc:299] Negotiation successful
0923 20:46:27.962098 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":261,"threads_started":1}
I20260923 20:46:27.963534 8339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:27.943016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:32783 (local address 127.5.71.131:34675)
0923 20:46:27.943714 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:27.943752 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:27.945562 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:27.946477 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:27.946490 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:27.947366 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:27.948112 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:27.948144 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.949454 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.949459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:27.951017 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:27.951036 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:27.952193 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:27.952199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:27.952427 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:27.953373 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:27.953398 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:27.955901 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:27.958937 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:27.958945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:27.958960 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:27.959383 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:27.959916 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:27.959921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:27.959924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:27.960181 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:27.960910 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:27.960919 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:27.961308 (+ 389us) client_negotiation.cc:769] Sending connection context
0923 20:46:27.961710 (+ 402us) client_negotiation.cc:240] Negotiation successful
0923 20:46:27.962313 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":278,"threads_started":1}
I20260923 20:46:27.965106 8336 heartbeater.cc:347] Connected to a master server at 127.5.71.190:32783
I20260923 20:46:27.965425 8336 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:27.966221 8336 heartbeater.cc:511] Master 127.5.71.190:32783 requested a full tablet report, sending...
I20260923 20:46:27.967774 7880 ts_manager.cc:194] Registered new tserver with Master: 52c02eabb48b442abe8739b9fc1ddbed (127.5.71.131:39675)
I20260923 20:46:27.968642 7880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:34675
I20260923 20:46:27.975606 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 20:46:28.010361 8338 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:28.007571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51672 (local address 127.5.71.190:32783)
0923 20:46:28.007752 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:28.007756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:28.008332 (+ 576us) server_negotiation.cc:407] Connection header received
0923 20:46:28.008554 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:28.008559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:28.008628 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:28.008740 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:28.010095 (+ 1355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51672: BlockingRecv error: recv got EOF from 127.0.0.1:51672 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:46:28.013593 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7941
W20260923 20:46:28.016997 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7941: Not authorized: Client connection negotiation failed: client connection to 127.5.71.129:39591: server requires authentication, but client does not have Kerberos credentials available
I20260923 20:46:28.017146 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 7941
I20260923 20:46:28.033246 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8075
W20260923 20:46:28.036031 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8075: Not authorized: Client connection negotiation failed: client connection to 127.5.71.130:38835: server requires authentication, but client does not have Kerberos credentials available
I20260923 20:46:28.036234 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8075
I20260923 20:46:28.052189 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8208
W20260923 20:46:28.055927 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8208: Not authorized: Client connection negotiation failed: client connection to 127.5.71.131:39675: server requires authentication, but client does not have Kerberos credentials available
I20260923 20:46:28.056105 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8208
I20260923 20:46:28.072932 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 7848
I20260923 20:46:28.074280 7910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:28.180405 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 7848
2026-09-23T20:46:28Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3902 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:28 dist-test-slave-dn29 krb5kdc[8354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:28 dist-test-slave-dn29 krb5kdc[8354](info): set up 2 sockets
Sep 23 20:46:28 dist-test-slave-dn29 krb5kdc[8354](info): commencing operation
krb5kdc: starting...
W20260923 20:46:29.276625 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:29 dist-test-slave-dn29 krb5kdc[8354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:29.485224 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45865
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:29.824029 8367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:29.824330 8367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:29.824478 8367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:29.832705 8367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:29.832837 8367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:29.832921 8367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:29.833007 8367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:29.833081 8367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:29.843312 8367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:29.844825 8367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:29.846282 8367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:29.858667 8372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:29.858738 8373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:29.858696 8375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:29.859586 8367 server_base.cc:1034] running on GCE node
I20260923 20:46:29.860466 8367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:29.861881 8367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:29.863096 8367 hybrid_clock.cc:648] HybridClock initialized: now 1790196389863044 us; error 40 us; skew 500 ppm
I20260923 20:46:29.863483 8367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:29 dist-test-slave-dn29 krb5kdc[8354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196389, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:29.873509 8367 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:29.875232 8367 webserver.cc:460] Webserver started at http://127.5.71.190:39165/ using document root <none> and password file <none>
I20260923 20:46:29.876035 8367 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:29.876144 8367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:29.876412 8367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:29.879438 8367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "544d588580b745b88fe3ffb0ec127792"
format_stamp: "Formatted at 2026-09-23 20:46:29 on dist-test-slave-dn29"
server_key: "b39cbdf4a0a28c4d3eb9ac287b05c466"
server_key_iv: "df2291f7aba89fb47837615d6f0d1021"
server_key_version: "encryptionkey@0"
I20260923 20:46:29.880282 8367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "544d588580b745b88fe3ffb0ec127792"
format_stamp: "Formatted at 2026-09-23 20:46:29 on dist-test-slave-dn29"
server_key: "b39cbdf4a0a28c4d3eb9ac287b05c466"
server_key_iv: "df2291f7aba89fb47837615d6f0d1021"
server_key_version: "encryptionkey@0"
I20260923 20:46:29.885850 8367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:29.889722 8382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:29.891378 8367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:46:29.891582 8367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "544d588580b745b88fe3ffb0ec127792"
format_stamp: "Formatted at 2026-09-23 20:46:29 on dist-test-slave-dn29"
server_key: "b39cbdf4a0a28c4d3eb9ac287b05c466"
server_key_iv: "df2291f7aba89fb47837615d6f0d1021"
server_key_version: "encryptionkey@0"
I20260923 20:46:29.891796 8367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:29.906003 8367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:29.910709 8367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:29.911041 8367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:29.930416 8367 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45865
I20260923 20:46:29.930502 8433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45865 every 8 connection(s)
I20260923 20:46:29.931969 8367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:29.935984 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:29.936084 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8367
I20260923 20:46:29.936327 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:29.937021 5406 external_mini_cluster.cc:1428] Setting key 99b697de8a88a66714938602512fee4c
I20260923 20:46:29.950553 8434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792: Bootstrap starting.
Sep 23 20:46:29 dist-test-slave-dn29 krb5kdc[8354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:29.954499 8434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:29.955611 8434 log.cc:824] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:29.958539 8434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792: No bootstrap required, opened a new log
I20260923 20:46:29.961585 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:29.938959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.5.71.190:45865)
0923 20:46:29.939911 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:29.939924 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:29.939967 (+ 43us) server_negotiation.cc:407] Connection header received
0923 20:46:29.942220 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:29.942250 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:29.942709 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:29.943112 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:29.945013 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:29.946391 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:29.948292 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:29.949086 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:29.949223 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:29.954070 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:29.954079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:29.954096 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:29.954139 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:29.957305 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:29.958084 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:29.958091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:29.958098 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:29.958262 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:29.958662 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:29.958666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:29.958672 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:29.959224 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:29.959563 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:29.960050 (+ 487us) server_negotiation.cc:299] Negotiation successful
0923 20:46:29.960520 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":436,"threads_started":1}
I20260923 20:46:29.965613 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:46:29.966966 8434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } }
I20260923 20:46:29.967340 8434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:29.967450 8434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544d588580b745b88fe3ffb0ec127792, State: Initialized, Role: FOLLOWER
I20260923 20:46:29.968428 8434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } }
I20260923 20:46:29.968739 8434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:29.968853 8434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:29.969019 8434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:29.971151 8434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } }
I20260923 20:46:29.971841 8439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:29.971872 8434 leader_election.cc:302] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 544d588580b745b88fe3ffb0ec127792; no voters:
I20260923 20:46:29.972059 8439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } }
I20260923 20:46:29.972944 8439 leader_election.cc:302] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 544d588580b745b88fe3ffb0ec127792; no voters:
I20260923 20:46:29.973629 8439 leader_election.cc:288] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260923 20:46:29.973846 8442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:29.974104 8442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [term 1 LEADER]: Becoming Leader. State: Replica: 544d588580b745b88fe3ffb0ec127792, State: Running, Role: LEADER
I20260923 20:46:29.974290 8434 leader_election.cc:288] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:29.974850 8442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } }
I20260923 20:46:29.976723 8434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:29.977178 8441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 544d588580b745b88fe3ffb0ec127792. Latest consensus state: current_term: 1 leader_uuid: "544d588580b745b88fe3ffb0ec127792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } } }
I20260923 20:46:29.977509 8441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:29.978092 8443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "544d588580b745b88fe3ffb0ec127792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "544d588580b745b88fe3ffb0ec127792" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45865 } } }
I20260923 20:46:29.978422 8445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:29.978423 8443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:30.001580 8445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:30.018052 8445 catalog_manager.cc:1347] Generated new cluster ID: e2c5347f85cd455687ef12f9c6786fef
I20260923 20:46:30.018195 8445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:30.026990 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:30.018706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.5.71.190:45865)
0923 20:46:30.019052 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:30.019061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:30.019739 (+ 678us) server_negotiation.cc:407] Connection header received
0923 20:46:30.020003 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:30.020015 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:30.020083 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:30.020200 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:30.021286 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:30.022100 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:30.023584 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:30.024161 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:30.024323 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:30.025470 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:30.025476 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:30.025482 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:30.026187 (+ 705us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:30.026259 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:30.026287 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:30.026410 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:30.026541 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 20:46:30.026644 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260923 20:46:30.035426 8445 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:30.037354 8445 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:30.045871 8445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 544d588580b745b88fe3ffb0ec127792: Generated new TSK 0
I20260923 20:46:30.046808 8445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:46:30.055485 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8367
I20260923 20:46:30.056811 8429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:30.150177 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8367
2026-09-23T20:46:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1964 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:30 dist-test-slave-dn29 krb5kdc[8469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:30 dist-test-slave-dn29 krb5kdc[8469](info): set up 2 sockets
Sep 23 20:46:30 dist-test-slave-dn29 krb5kdc[8469](info): commencing operation
krb5kdc: starting...
W20260923 20:46:31.288661 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:31 dist-test-slave-dn29 krb5kdc[8469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:31.500847 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43273
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:31.842137 8482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:31.842523 8482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:31.842692 8482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:31.850085 8482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:31.850256 8482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:31.850369 8482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:31.850472 8482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:31.850548 8482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:31.860649 8482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:31.862044 8482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:31.863438 8482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:31.875661 8488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:31.875643 8487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:31.875626 8490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:31.876715 8482 server_base.cc:1034] running on GCE node
I20260923 20:46:31.877585 8482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:31.879292 8482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:31.880565 8482 hybrid_clock.cc:648] HybridClock initialized: now 1790196391880516 us; error 35 us; skew 500 ppm
I20260923 20:46:31.880966 8482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:31 dist-test-slave-dn29 krb5kdc[8469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196391, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:31.884601 8482 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:31.886404 8482 webserver.cc:460] Webserver started at http://127.5.71.190:36927/ using document root <none> and password file <none>
I20260923 20:46:31.887163 8482 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:31.887336 8482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:31.887830 8482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:31.891211 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "a38fbb882f1e4330a916f2487b0a099d"
format_stamp: "Formatted at 2026-09-23 20:46:31 on dist-test-slave-dn29"
server_key: "2d7c9bb3c4153f367685751425bb0a66"
server_key_iv: "5d955b9d92ab34ca734e6a4fcb10234b"
server_key_version: "encryptionkey@0"
I20260923 20:46:31.892203 8482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "a38fbb882f1e4330a916f2487b0a099d"
format_stamp: "Formatted at 2026-09-23 20:46:31 on dist-test-slave-dn29"
server_key: "2d7c9bb3c4153f367685751425bb0a66"
server_key_iv: "5d955b9d92ab34ca734e6a4fcb10234b"
server_key_version: "encryptionkey@0"
I20260923 20:46:31.898203 8482 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:46:31.902107 8497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:31.903663 8482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:46:31.903977 8482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "a38fbb882f1e4330a916f2487b0a099d"
format_stamp: "Formatted at 2026-09-23 20:46:31 on dist-test-slave-dn29"
server_key: "2d7c9bb3c4153f367685751425bb0a66"
server_key_iv: "5d955b9d92ab34ca734e6a4fcb10234b"
server_key_version: "encryptionkey@0"
I20260923 20:46:31.904206 8482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:31.925442 8482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:31.929955 8482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:31.930239 8482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:31.948704 8482 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:43273
I20260923 20:46:31.948724 8548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:43273 every 8 connection(s)
I20260923 20:46:31.950168 8482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:31.950928 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8482
I20260923 20:46:31.951153 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:31.952088 5406 external_mini_cluster.cc:1428] Setting key 0756b199ee3f151c5caf5f3e0f91204c
I20260923 20:46:31.954849 8549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:46:31 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:31.965857 8549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d: Bootstrap starting.
I20260923 20:46:31.969317 8549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:31.970424 8549 log.cc:824] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:31.973955 8549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d: No bootstrap required, opened a new log
I20260923 20:46:31.974635 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:31.954146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.5.71.190:43273)
0923 20:46:31.954993 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:31.955012 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:31.955064 (+ 52us) server_negotiation.cc:407] Connection header received
0923 20:46:31.956802 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:31.956826 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:31.957227 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:31.957668 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:31.959101 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:31.960524 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:31.961736 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:31.962440 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:31.962648 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:31.966948 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:31.966956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:31.966971 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:31.967008 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:31.970209 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:31.971016 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:31.971026 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:31.971041 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:31.971309 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:31.971738 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:31.971743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:31.971746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:31.972347 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:31.972564 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:31.973034 (+ 470us) server_negotiation.cc:299] Negotiation successful
0923 20:46:31.973429 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":350,"threads_started":1}
I20260923 20:46:31.982264 8549 raft_consensus.cc:348] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38fbb882f1e4330a916f2487b0a099d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43273 } }
I20260923 20:46:31.982690 8549 raft_consensus.cc:374] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:31.982807 8549 raft_consensus.cc:729] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38fbb882f1e4330a916f2487b0a099d, State: Initialized, Role: FOLLOWER
I20260923 20:46:31.983367 8549 consensus_queue.cc:260] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38fbb882f1e4330a916f2487b0a099d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43273 } }
I20260923 20:46:31.983611 8549 raft_consensus.cc:388] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:31.983767 8549 raft_consensus.cc:482] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:31.984017 8549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:31.986537 8549 raft_consensus.cc:504] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38fbb882f1e4330a916f2487b0a099d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43273 } }
I20260923 20:46:31.987087 8549 leader_election.cc:302] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a38fbb882f1e4330a916f2487b0a099d; no voters:
I20260923 20:46:31.987715 8549 leader_election.cc:288] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:31.987937 8554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:31.988744 8554 raft_consensus.cc:686] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [term 1 LEADER]: Becoming Leader. State: Replica: a38fbb882f1e4330a916f2487b0a099d, State: Running, Role: LEADER
I20260923 20:46:31.989392 8554 consensus_queue.cc:237] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38fbb882f1e4330a916f2487b0a099d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43273 } }
I20260923 20:46:31.989996 8549 sys_catalog.cc:564] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:31.992880 8555 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a38fbb882f1e4330a916f2487b0a099d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38fbb882f1e4330a916f2487b0a099d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43273 } } }
I20260923 20:46:31.993386 8555 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:31.993083 8556 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a38fbb882f1e4330a916f2487b0a099d. Latest consensus state: current_term: 1 leader_uuid: "a38fbb882f1e4330a916f2487b0a099d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a38fbb882f1e4330a916f2487b0a099d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43273 } } }
I20260923 20:46:31.994153 8556 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d [sys.catalog]: This master's current role is: LEADER
W20260923 20:46:32.004277 8570 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:46:32.004453 8570 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:46:32.004736 8560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:32.006420 8560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:32.015736 8560 catalog_manager.cc:1347] Generated new cluster ID: 669fb57b27d0468385081dce3611781a
I20260923 20:46:32.015949 8560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:32.044567 8560 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:32.046550 8560 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:32.065719 8560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a38fbb882f1e4330a916f2487b0a099d: Generated new TSK 0
I20260923 20:46:32.066907 8560 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:32.120213 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43273
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:32.460631 8575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:32.460953 8575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:32.461048 8575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:32.468386 8575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:32.468513 8575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:32.468631 8575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:32.479002 8575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:32.480413 8575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:32.481706 8575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:32.496268 8580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:32.496373 8581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:32.496551 8583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:32.497846 8575 server_base.cc:1034] running on GCE node
I20260923 20:46:32.498574 8575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:32.499752 8575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:32.501089 8575 hybrid_clock.cc:648] HybridClock initialized: now 1790196392501034 us; error 44 us; skew 500 ppm
I20260923 20:46:32.501511 8575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:32 dist-test-slave-dn29 krb5kdc[8469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196392, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:32.505249 8575 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:32.506754 8575 webserver.cc:460] Webserver started at http://127.5.71.129:40479/ using document root <none> and password file <none>
I20260923 20:46:32.507534 8575 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:32.507640 8575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:32.507979 8575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:32.511118 8575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "d483e92a41d848a499e753375afa3e85"
format_stamp: "Formatted at 2026-09-23 20:46:32 on dist-test-slave-dn29"
server_key: "2c5ca071256ed4bc62d51f81d96108d5"
server_key_iv: "6c091943daa4238499a283874a5f20a7"
server_key_version: "encryptionkey@0"
I20260923 20:46:32.512264 8575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "d483e92a41d848a499e753375afa3e85"
format_stamp: "Formatted at 2026-09-23 20:46:32 on dist-test-slave-dn29"
server_key: "2c5ca071256ed4bc62d51f81d96108d5"
server_key_iv: "6c091943daa4238499a283874a5f20a7"
server_key_version: "encryptionkey@0"
I20260923 20:46:32.519652 8575 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260923 20:46:32.523602 8590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:32.525446 8575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:32.525714 8575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "d483e92a41d848a499e753375afa3e85"
format_stamp: "Formatted at 2026-09-23 20:46:32 on dist-test-slave-dn29"
server_key: "2c5ca071256ed4bc62d51f81d96108d5"
server_key_iv: "6c091943daa4238499a283874a5f20a7"
server_key_version: "encryptionkey@0"
I20260923 20:46:32.525938 8575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:32.544080 8575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:32.549114 8575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:32.549469 8575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:32.550557 8575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:32.552186 8575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:32.552279 8575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:32.552417 8575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:32.552495 8575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:32.590770 8575 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:39237
I20260923 20:46:32.590924 8702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:39237 every 8 connection(s)
I20260923 20:46:32.592335 8575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:32.601452 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8575
I20260923 20:46:32.601682 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:32.602384 5406 external_mini_cluster.cc:1428] Setting key 06768a5b0f44fe9648ff35abf34b22ff
Sep 23 20:46:32 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196392, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:32.616309 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:32.595461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:34053 (local address 127.5.71.190:43273)
0923 20:46:32.595970 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:32.595975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:32.597196 (+ 1221us) server_negotiation.cc:407] Connection header received
0923 20:46:32.599198 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:32.599203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:32.599298 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:32.599422 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:32.601153 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:32.602141 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:32.604237 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:32.604847 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:32.604959 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:32.609786 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:32.609792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:32.609796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:32.609834 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:32.612822 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:32.613690 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:32.613697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:32.613704 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:32.613841 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:32.614348 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:32.614358 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:32.614361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:32.614646 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:32.614819 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:32.615619 (+ 800us) server_negotiation.cc:299] Negotiation successful
0923 20:46:32.615897 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":265,"threads_started":1}
I20260923 20:46:32.617434 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:32.596153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43273 (local address 127.5.71.129:34053)
0923 20:46:32.597032 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:32.597071 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:32.598901 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:32.599679 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:32.599688 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:32.600275 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:32.600886 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:32.600905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:32.602321 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:32.602327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:32.604088 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:32.604101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:32.605115 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:32.605123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:32.605484 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:32.606276 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:32.606315 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:32.609464 (+ 3149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:32.613012 (+ 3548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:32.613023 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:32.613045 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:32.613541 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:32.613970 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:32.613975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:32.613979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:32.614212 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:32.614788 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:32.614797 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:32.615208 (+ 411us) client_negotiation.cc:769] Sending connection context
0923 20:46:32.615500 (+ 292us) client_negotiation.cc:240] Negotiation successful
0923 20:46:32.616179 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":411,"threads_started":1}
I20260923 20:46:32.619341 8703 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43273
I20260923 20:46:32.619741 8703 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:32.620821 8703 heartbeater.cc:511] Master 127.5.71.190:43273 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:32.623378 8514 ts_manager.cc:194] Registered new tserver with Master: d483e92a41d848a499e753375afa3e85 (127.5.71.129:39237)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:32.625101 8514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:34053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:32.653105 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43273
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:32.989758 8709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:32.990068 8709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:32.990222 8709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:32.997972 8709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:32.998095 8709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:32.998320 8709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:33.008801 8709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:33.010262 8709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:33.011613 8709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:33.025329 8715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:33.025388 8717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:33.025365 8714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:33.026157 8709 server_base.cc:1034] running on GCE node
I20260923 20:46:33.026832 8709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:33.027886 8709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:33.029124 8709 hybrid_clock.cc:648] HybridClock initialized: now 1790196393029070 us; error 45 us; skew 500 ppm
I20260923 20:46:33.029481 8709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:33.033166 8709 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:33.034945 8709 webserver.cc:460] Webserver started at http://127.5.71.130:43435/ using document root <none> and password file <none>
I20260923 20:46:33.035853 8709 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:33.036028 8709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:33.036397 8709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:33.040035 8709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "ce1403cc9f1f44daab35fb67d5bc1a5c"
format_stamp: "Formatted at 2026-09-23 20:46:33 on dist-test-slave-dn29"
server_key: "860ab6354bf36c3a91c681ece06c5baa"
server_key_iv: "6bc6e0205c616c830fdef69eec962fa5"
server_key_version: "encryptionkey@0"
I20260923 20:46:33.040987 8709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "ce1403cc9f1f44daab35fb67d5bc1a5c"
format_stamp: "Formatted at 2026-09-23 20:46:33 on dist-test-slave-dn29"
server_key: "860ab6354bf36c3a91c681ece06c5baa"
server_key_iv: "6bc6e0205c616c830fdef69eec962fa5"
server_key_version: "encryptionkey@0"
I20260923 20:46:33.047196 8709 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:46:33.051896 8724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:33.053670 8709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:46:33.053908 8709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "ce1403cc9f1f44daab35fb67d5bc1a5c"
format_stamp: "Formatted at 2026-09-23 20:46:33 on dist-test-slave-dn29"
server_key: "860ab6354bf36c3a91c681ece06c5baa"
server_key_iv: "6bc6e0205c616c830fdef69eec962fa5"
server_key_version: "encryptionkey@0"
I20260923 20:46:33.054159 8709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:33.076189 8709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:33.081138 8709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:33.081487 8709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:33.082577 8709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:33.084213 8709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:33.084321 8709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:33.084478 8709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:33.084551 8709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:33.121255 8709 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:39059
I20260923 20:46:33.121348 8836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:39059 every 8 connection(s)
I20260923 20:46:33.122615 8709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:33.124380 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8709
I20260923 20:46:33.124552 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:33.125082 5406 external_mini_cluster.cc:1428] Setting key ac209c1f61d94610bbecabc6ca467180
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:33.145813 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.125552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:45121 (local address 127.5.71.190:43273)
0923 20:46:33.126042 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:33.126048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:33.127272 (+ 1224us) server_negotiation.cc:407] Connection header received
0923 20:46:33.130232 (+ 2960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:33.130238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:33.130329 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:33.130466 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:33.132833 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.133552 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.135204 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.135645 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.135788 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.139727 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:33.139732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:33.139736 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:33.139770 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:33.142531 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.143341 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.143348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.143357 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.143473 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.143972 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.143977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.143982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.144320 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:33.144477 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:33.145187 (+ 710us) server_negotiation.cc:299] Negotiation successful
0923 20:46:33.145417 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":263,"threads_started":1}
I20260923 20:46:33.146822 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.126236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43273 (local address 127.5.71.130:45121)
0923 20:46:33.127106 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:33.127155 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:33.129881 (+ 2726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:33.130913 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:33.130926 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:33.131740 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:33.132626 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:33.132655 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.133728 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.133734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:33.135052 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:33.135065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.135928 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.135937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:33.136202 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.136909 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:33.136937 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:33.139461 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:33.142687 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:33.142697 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:33.142724 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:33.143162 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:33.143585 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:33.143590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:33.143594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:33.143850 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:33.144449 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:33.144458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:33.144827 (+ 369us) client_negotiation.cc:769] Sending connection context
0923 20:46:33.145097 (+ 270us) client_negotiation.cc:240] Negotiation successful
0923 20:46:33.145659 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":320,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:33.148362 8837 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43273
I20260923 20:46:33.148780 8837 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:33.149762 8837 heartbeater.cc:511] Master 127.5.71.190:43273 requested a full tablet report, sending...
I20260923 20:46:33.151485 8514 ts_manager.cc:194] Registered new tserver with Master: ce1403cc9f1f44daab35fb67d5bc1a5c (127.5.71.130:39059)
I20260923 20:46:33.152745 8514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:45121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:33.176201 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43273
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:33.516336 8843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:33.516630 8843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:33.516754 8843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:33.524181 8843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:33.524293 8843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:33.524457 8843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:33.534695 8843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:33.536329 8843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:33.537650 8843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:33.552567 8848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:33.552668 8849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:33.552577 8851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:33.553925 8843 server_base.cc:1034] running on GCE node
I20260923 20:46:33.554620 8843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:33.555703 8843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:33.557003 8843 hybrid_clock.cc:648] HybridClock initialized: now 1790196393556957 us; error 42 us; skew 500 ppm
I20260923 20:46:33.557403 8843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:33.561213 8843 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:33.563020 8843 webserver.cc:460] Webserver started at http://127.5.71.131:34985/ using document root <none> and password file <none>
I20260923 20:46:33.563892 8843 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:33.564055 8843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:33.564401 8843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:33.567902 8843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "69a6d7188e0f42cd8836609aa8975f65"
format_stamp: "Formatted at 2026-09-23 20:46:33 on dist-test-slave-dn29"
server_key: "b87a2aa83535fc9defc866cdb3daf632"
server_key_iv: "c83a37907d968793ba9f5971df700f4b"
server_key_version: "encryptionkey@0"
I20260923 20:46:33.568823 8843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "69a6d7188e0f42cd8836609aa8975f65"
format_stamp: "Formatted at 2026-09-23 20:46:33 on dist-test-slave-dn29"
server_key: "b87a2aa83535fc9defc866cdb3daf632"
server_key_iv: "c83a37907d968793ba9f5971df700f4b"
server_key_version: "encryptionkey@0"
I20260923 20:46:33.574939 8843 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260923 20:46:33.579073 8858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:33.580834 8843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:33.581040 8843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "69a6d7188e0f42cd8836609aa8975f65"
format_stamp: "Formatted at 2026-09-23 20:46:33 on dist-test-slave-dn29"
server_key: "b87a2aa83535fc9defc866cdb3daf632"
server_key_iv: "c83a37907d968793ba9f5971df700f4b"
server_key_version: "encryptionkey@0"
I20260923 20:46:33.581284 8843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:33.612095 8843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:33.617004 8843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:33.617374 8843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:33.618495 8843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:33.620000 8843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:33.620114 8843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:33.620278 8843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:33.620362 8843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:33.628401 8703 heartbeater.cc:503] Master 127.5.71.190:43273 was elected leader, sending a full tablet report...
I20260923 20:46:33.660434 8843 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:45649
I20260923 20:46:33.660528 8970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:45649 every 8 connection(s)
I20260923 20:46:33.661836 8843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:33.667353 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 8843
I20260923 20:46:33.667546 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:33.668184 5406 external_mini_cluster.cc:1428] Setting key 925000821f1fd6b7c5e24ce799f0dc18
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:33.683517 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.664828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:49027 (local address 127.5.71.190:43273)
0923 20:46:33.665359 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:33.665364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:33.666259 (+ 895us) server_negotiation.cc:407] Connection header received
0923 20:46:33.668538 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:33.668544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:33.668639 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:33.668776 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:33.670398 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.671305 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.673141 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.673573 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.673722 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.677370 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:33.677375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:33.677379 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:33.677413 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:33.680215 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.681005 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.681009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.681012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.681131 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.681603 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.681616 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.681619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.681908 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:33.682101 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:33.682860 (+ 759us) server_negotiation.cc:299] Negotiation successful
0923 20:46:33.683098 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":294,"threads_started":1}
I20260923 20:46:33.684685 8974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.665329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43273 (local address 127.5.71.131:49027)
0923 20:46:33.666076 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:33.666121 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:33.668181 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:33.669038 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:33.669047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:33.669594 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:33.670203 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:33.670221 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.671477 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.671483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:33.672977 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:33.672990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.673816 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.673824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:33.674077 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.674732 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:33.674761 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:33.677132 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:33.680367 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:33.680377 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:33.680396 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:33.680853 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:33.681260 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:33.681265 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:33.681269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:33.681470 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:33.682072 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:33.682081 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:33.682454 (+ 373us) client_negotiation.cc:769] Sending connection context
0923 20:46:33.682741 (+ 287us) client_negotiation.cc:240] Negotiation successful
0923 20:46:33.683338 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":285,"threads_started":1}
I20260923 20:46:33.686344 8971 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43273
I20260923 20:46:33.686802 8971 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:33.687651 8971 heartbeater.cc:511] Master 127.5.71.190:43273 requested a full tablet report, sending...
I20260923 20:46:33.689333 8514 ts_manager.cc:194] Registered new tserver with Master: 69a6d7188e0f42cd8836609aa8975f65 (127.5.71.131:45649)
I20260923 20:46:33.690230 8514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:49027
I20260923 20:46:33.696306 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:33.748937 8973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.734655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.5.71.190:43273)
0923 20:46:33.734913 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:33.734918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:33.735299 (+ 381us) server_negotiation.cc:407] Connection header received
0923 20:46:33.735450 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:33.735455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:33.735525 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:33.735624 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:33.737172 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.737983 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.739247 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.739717 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.739919 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.743897 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:33.743903 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:33.743907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:33.743945 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:33.746620 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.747205 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.747209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.747211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.747332 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.747671 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.747675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.747677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.748068 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:33.748246 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:33.748518 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 20:46:33.748687 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260923 20:46:33.749505 8514 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54346
I20260923 20:46:33.751408 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8575
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:33.768779 8983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.752528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.5.71.129:39237)
0923 20:46:33.753151 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:33.753158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:33.753181 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:46:33.753234 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:33.753240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:33.753463 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:33.753589 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:33.754948 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.756059 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.757316 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.758002 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.758101 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.762486 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:33.762498 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:33.762515 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:33.762554 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:33.765698 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.766405 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.766412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.766414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.766535 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.766982 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.766986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.766988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.767418 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:33.767751 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:33.768043 (+ 292us) server_negotiation.cc:299] Negotiation successful
0923 20:46:33.768260 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":335,"threads_started":1}
W20260923 20:46:33.770213 8698 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54708
W20260923 20:46:33.771637 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8575: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:33.771899 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8575
I20260923 20:46:33.787518 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8709
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:33.805038 8984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.788574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.5.71.130:39059)
0923 20:46:33.789146 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:33.789155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:33.789179 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:46:33.789248 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:33.789255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:33.789465 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:33.789656 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:33.790916 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.792113 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.793451 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.794009 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.794192 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.798666 (+ 4474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:33.798678 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:33.798695 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:33.798739 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:33.801735 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.802508 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.802517 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.802519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.802652 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.803064 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.803072 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.803076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.803460 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:33.803845 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:33.804307 (+ 462us) server_negotiation.cc:299] Negotiation successful
0923 20:46:33.804612 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":293,"threads_started":1}
W20260923 20:46:33.806428 8832 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47002
W20260923 20:46:33.807643 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8709: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:33.807848 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8709
I20260923 20:46:33.824905 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8843
Sep 23 20:46:33 dist-test-slave-dn29 krb5kdc[8469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196393, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:33.843411 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:33.826041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44962 (local address 127.5.71.131:45649)
0923 20:46:33.826855 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:33.826863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:33.826894 (+ 31us) server_negotiation.cc:407] Connection header received
0923 20:46:33.826963 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:33.826971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:33.827208 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:33.827354 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:33.828774 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.829942 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.831204 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:33.832163 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:33.832286 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:33.836966 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:33.836981 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:33.837000 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:33.837049 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:33.840482 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.841162 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.841170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.841172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.841325 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:33.841786 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:33.841790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:33.841792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:33.842176 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:33.842461 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:33.842729 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 20:46:33.842954 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":469,"threads_started":1}
W20260923 20:46:33.844882 8966 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44962
W20260923 20:46:33.846050 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8843: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:46:33.846349 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8843
I20260923 20:46:33.862239 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 8482
I20260923 20:46:33.863443 8544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:33.968212 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 8482
2026-09-23T20:46:33Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3824 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:34 dist-test-slave-dn29 krb5kdc[8989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:34 dist-test-slave-dn29 krb5kdc[8989](info): set up 2 sockets
Sep 23 20:46:34 dist-test-slave-dn29 krb5kdc[8989](info): commencing operation
krb5kdc: starting...
W20260923 20:46:35.120676 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.005s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:35 dist-test-slave-dn29 krb5kdc[8989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:35.328205 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:34321
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:34321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:35.667062 9002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:35.667418 9002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:35.667582 9002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:35.675235 9002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:35.675369 9002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:35.675472 9002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:35.675577 9002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:35.675670 9002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:35.686194 9002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:34321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:34321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:35.687650 9002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:35.689070 9002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:35.701960 9007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:35.702035 9010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:35.702003 9008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:35.702950 9002 server_base.cc:1034] running on GCE node
I20260923 20:46:35.703846 9002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:35.705442 9002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:35.706801 9002 hybrid_clock.cc:648] HybridClock initialized: now 1790196395706729 us; error 70 us; skew 500 ppm
I20260923 20:46:35.707187 9002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:35 dist-test-slave-dn29 krb5kdc[8989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196395, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:35.717161 9002 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:35.719136 9002 webserver.cc:460] Webserver started at http://127.5.71.190:43675/ using document root <none> and password file <none>
I20260923 20:46:35.720139 9002 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:35.720309 9002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:35.720680 9002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:35.724180 9002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "e7b4080664354aa5927e00227242df80"
format_stamp: "Formatted at 2026-09-23 20:46:35 on dist-test-slave-dn29"
server_key: "ac2f895b64e372855efcc3fbc0a67bf7"
server_key_iv: "1dff41844f691062d6d06973b7736696"
server_key_version: "encryptionkey@0"
I20260923 20:46:35.725102 9002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "e7b4080664354aa5927e00227242df80"
format_stamp: "Formatted at 2026-09-23 20:46:35 on dist-test-slave-dn29"
server_key: "ac2f895b64e372855efcc3fbc0a67bf7"
server_key_iv: "1dff41844f691062d6d06973b7736696"
server_key_version: "encryptionkey@0"
I20260923 20:46:35.730933 9002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260923 20:46:35.734915 9017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:35.736661 9002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:35.736886 9002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "e7b4080664354aa5927e00227242df80"
format_stamp: "Formatted at 2026-09-23 20:46:35 on dist-test-slave-dn29"
server_key: "ac2f895b64e372855efcc3fbc0a67bf7"
server_key_iv: "1dff41844f691062d6d06973b7736696"
server_key_version: "encryptionkey@0"
I20260923 20:46:35.737123 9002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:35.761906 9002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:35.766543 9002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:35.766880 9002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:35.786401 9002 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:34321
I20260923 20:46:35.786434 9068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:34321 every 8 connection(s)
I20260923 20:46:35.787822 9002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:35.789070 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9002
I20260923 20:46:35.789338 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:35.790125 5406 external_mini_cluster.cc:1428] Setting key 8605a3714ec958af74d6e9d1ea8c51dd
I20260923 20:46:35.792727 9069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:46:35 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:35.805893 9069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80: Bootstrap starting.
I20260923 20:46:35.810303 9069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:35.811529 9069 log.cc:824] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:35.815246 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:35.792195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49506 (local address 127.5.71.190:34321)
0923 20:46:35.793022 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:35.793033 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:35.793071 (+ 38us) server_negotiation.cc:407] Connection header received
0923 20:46:35.794660 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:35.794680 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:35.795079 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:35.795415 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:35.797265 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:35.798594 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:35.800353 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:35.801120 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:35.801295 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:35.805930 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:35.805942 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:35.805965 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:35.806023 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:35.810051 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:35.810860 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:35.810869 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:35.810878 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:35.811072 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:35.811562 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:35.811567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:35.811570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:35.812370 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:35.812600 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:35.813298 (+ 698us) server_negotiation.cc:299] Negotiation successful
0923 20:46:35.813796 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":431,"threads_started":1}
I20260923 20:46:35.815534 9069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80: No bootstrap required, opened a new log
I20260923 20:46:35.824327 9069 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4080664354aa5927e00227242df80" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34321 } }
I20260923 20:46:35.824939 9069 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:35.825057 9069 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b4080664354aa5927e00227242df80, State: Initialized, Role: FOLLOWER
I20260923 20:46:35.825603 9069 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4080664354aa5927e00227242df80" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34321 } }
I20260923 20:46:35.825820 9069 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:35.825928 9069 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:35.826066 9069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:35.828435 9069 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4080664354aa5927e00227242df80" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34321 } }
I20260923 20:46:35.828928 9069 leader_election.cc:302] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7b4080664354aa5927e00227242df80; no voters:
I20260923 20:46:35.829520 9069 leader_election.cc:288] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:35.829681 9074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:35.830952 9074 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [term 1 LEADER]: Becoming Leader. State: Replica: e7b4080664354aa5927e00227242df80, State: Running, Role: LEADER
I20260923 20:46:35.831621 9074 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4080664354aa5927e00227242df80" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34321 } }
I20260923 20:46:35.832125 9069 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:35.834971 9075 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7b4080664354aa5927e00227242df80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4080664354aa5927e00227242df80" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34321 } } }
I20260923 20:46:35.836040 9075 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:35.835565 9076 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7b4080664354aa5927e00227242df80. Latest consensus state: current_term: 1 leader_uuid: "e7b4080664354aa5927e00227242df80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7b4080664354aa5927e00227242df80" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34321 } } }
I20260923 20:46:35.836174 9076 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:35.838312 9082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:35.844357 9082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:35.855093 9082 catalog_manager.cc:1347] Generated new cluster ID: a23e842e3cfe429981b12a12025397c6
I20260923 20:46:35.855266 9082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:35.882866 9082 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:35.884346 9082 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:35.903575 9082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7b4080664354aa5927e00227242df80: Generated new TSK 0
I20260923 20:46:35.904870 9082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:35.964785 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34321
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:36.298293 9095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:36.298599 9095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:36.298774 9095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:36.306309 9095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:36.306448 9095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:36.306645 9095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:36.316754 9095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:36.318131 9095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:36.319470 9095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:36.334379 9100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:36.334826 9103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:36.334862 9095 server_base.cc:1034] running on GCE node
W20260923 20:46:36.335258 9101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:36.336077 9095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:36.337059 9095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:36.338302 9095 hybrid_clock.cc:648] HybridClock initialized: now 1790196396338235 us; error 60 us; skew 500 ppm
I20260923 20:46:36.338711 9095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:36 dist-test-slave-dn29 krb5kdc[8989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:36.342054 9095 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:36.343652 9095 webserver.cc:460] Webserver started at http://127.5.71.129:37979/ using document root <none> and password file <none>
I20260923 20:46:36.344425 9095 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:36.344529 9095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:36.344867 9095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:36.347951 9095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "09bdab86ca2e4c4491bcebc151287229"
format_stamp: "Formatted at 2026-09-23 20:46:36 on dist-test-slave-dn29"
server_key: "39e6269adddf09590f6ce5112ede1762"
server_key_iv: "0ddcb1f8795a37bed562efc32478a18f"
server_key_version: "encryptionkey@0"
I20260923 20:46:36.348750 9095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "09bdab86ca2e4c4491bcebc151287229"
format_stamp: "Formatted at 2026-09-23 20:46:36 on dist-test-slave-dn29"
server_key: "39e6269adddf09590f6ce5112ede1762"
server_key_iv: "0ddcb1f8795a37bed562efc32478a18f"
server_key_version: "encryptionkey@0"
I20260923 20:46:36.354092 9095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:36.358083 9110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:36.359441 9095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260923 20:46:36.359612 9095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "09bdab86ca2e4c4491bcebc151287229"
format_stamp: "Formatted at 2026-09-23 20:46:36 on dist-test-slave-dn29"
server_key: "39e6269adddf09590f6ce5112ede1762"
server_key_iv: "0ddcb1f8795a37bed562efc32478a18f"
server_key_version: "encryptionkey@0"
I20260923 20:46:36.359890 9095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:36.377444 9095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:36.382107 9095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:36.382427 9095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:36.383474 9095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:36.384938 9095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:36.385042 9095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:36.385193 9095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:36.385277 9095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:36.419266 9095 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:39029
I20260923 20:46:36.419353 9222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:39029 every 8 connection(s)
I20260923 20:46:36.420675 9095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:36.424630 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9095
I20260923 20:46:36.424896 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:36.425647 5406 external_mini_cluster.cc:1428] Setting key 13cc0cb0f7f523732546cf3b04f43d48
Sep 23 20:46:36 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:36.444833 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:36.423465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:39233 (local address 127.5.71.190:34321)
0923 20:46:36.424022 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:36.424027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:36.425143 (+ 1116us) server_negotiation.cc:407] Connection header received
0923 20:46:36.427231 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:36.427237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:36.427334 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:36.427469 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:36.429242 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.429964 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.431574 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.432059 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.432235 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:36.436875 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:36.436885 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:36.436892 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:36.436942 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:36.441087 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:36.442196 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:36.442204 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:36.442211 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:36.442367 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:36.442923 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:36.442929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:36.442937 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:36.443216 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:36.443398 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:36.444226 (+ 828us) server_negotiation.cc:299] Negotiation successful
0923 20:46:36.444473 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":270,"threads_started":1}
I20260923 20:46:36.446204 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:36.424109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:34321 (local address 127.5.71.129:39233)
0923 20:46:36.424951 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:36.425005 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:36.426948 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:36.427733 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:36.427742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:36.428413 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:36.429037 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:36.429055 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.430129 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.430133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:36.431424 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:36.431436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.432350 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.432357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:36.432651 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:36.433576 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:36.433617 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:36.436561 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:36.441285 (+ 4724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:36.441294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:36.441314 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:36.442024 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:36.442523 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:36.442528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:36.442535 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:36.442744 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:36.443370 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:36.443379 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:36.443757 (+ 378us) client_negotiation.cc:769] Sending connection context
0923 20:46:36.444117 (+ 360us) client_negotiation.cc:240] Negotiation successful
0923 20:46:36.444697 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":354,"threads_started":1}
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:36.448326 9223 heartbeater.cc:347] Connected to a master server at 127.5.71.190:34321
I20260923 20:46:36.448642 9223 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:36.449584 9223 heartbeater.cc:511] Master 127.5.71.190:34321 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:36.452086 9034 ts_manager.cc:194] Registered new tserver with Master: 09bdab86ca2e4c4491bcebc151287229 (127.5.71.129:39029)
I20260923 20:46:36.453634 9034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:39233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:36.476428 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34321
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:36.812841 9229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:36.813140 9229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:36.813297 9229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:36.820845 9229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:36.820966 9229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:36.821138 9229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:36.831240 9229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:36.832690 9229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:36.834102 9229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:36.848707 9234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:36.848739 9235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:36.849093 9237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:36.849841 9229 server_base.cc:1034] running on GCE node
I20260923 20:46:36.850548 9229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:36.851717 9229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:36.853006 9229 hybrid_clock.cc:648] HybridClock initialized: now 1790196396852949 us; error 47 us; skew 500 ppm
I20260923 20:46:36.853367 9229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:36 dist-test-slave-dn29 krb5kdc[8989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:36.856848 9229 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:36.858541 9229 webserver.cc:460] Webserver started at http://127.5.71.130:40169/ using document root <none> and password file <none>
I20260923 20:46:36.859285 9229 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:36.859396 9229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:36.859679 9229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:36.862982 9229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "f0e44653eb9c4c95872b25ee805397f7"
format_stamp: "Formatted at 2026-09-23 20:46:36 on dist-test-slave-dn29"
server_key: "b7b76447fb637ab3c71627588cd83855"
server_key_iv: "17cb40c05370e717ea332de1e74efd0c"
server_key_version: "encryptionkey@0"
I20260923 20:46:36.864046 9229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "f0e44653eb9c4c95872b25ee805397f7"
format_stamp: "Formatted at 2026-09-23 20:46:36 on dist-test-slave-dn29"
server_key: "b7b76447fb637ab3c71627588cd83855"
server_key_iv: "17cb40c05370e717ea332de1e74efd0c"
server_key_version: "encryptionkey@0"
I20260923 20:46:36.869913 9229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:36.873778 9244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:36.875200 9229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:46:36.875379 9229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "f0e44653eb9c4c95872b25ee805397f7"
format_stamp: "Formatted at 2026-09-23 20:46:36 on dist-test-slave-dn29"
server_key: "b7b76447fb637ab3c71627588cd83855"
server_key_iv: "17cb40c05370e717ea332de1e74efd0c"
server_key_version: "encryptionkey@0"
I20260923 20:46:36.875591 9229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:36.900353 9229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:36.905125 9229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:36.905432 9229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:36.906435 9229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:36.907769 9229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:36.907900 9229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:36.908035 9229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:36.908094 9229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:36.942929 9229 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:41693
I20260923 20:46:36.943015 9356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:41693 every 8 connection(s)
I20260923 20:46:36.944298 9229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:36.950395 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9229
I20260923 20:46:36.950621 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:36.951143 5406 external_mini_cluster.cc:1428] Setting key 9d9d4e6dd1495099ed3c0d72a6f2127f
Sep 23 20:46:36 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196396, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:36.965617 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:36.946992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:56897 (local address 127.5.71.190:34321)
0923 20:46:36.947466 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:36.947471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:36.948744 (+ 1273us) server_negotiation.cc:407] Connection header received
0923 20:46:36.950653 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:36.950663 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:36.950759 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:36.950885 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:36.952528 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.953350 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.954940 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.955377 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.955511 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:36.959179 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:36.959184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:36.959187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:36.959217 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:36.961900 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:36.962851 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:36.962855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:36.962857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:36.962988 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:36.963485 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:36.963488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:36.963490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:36.963872 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:36.963982 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:36.965000 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0923 20:46:36.965223 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":270,"threads_started":1}
I20260923 20:46:36.967352 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:36.947639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:34321 (local address 127.5.71.130:56897)
0923 20:46:36.948471 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:36.948510 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:36.950357 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:36.951091 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:36.951099 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:36.951695 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:36.952352 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:36.952370 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.953503 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.953518 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:36.954777 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:36.954793 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:36.955601 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:36.955609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:36.955902 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:36.956612 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:36.956634 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:36.958945 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:36.962042 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:36.962055 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:36.962077 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:36.962725 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:36.963096 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:36.963103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:36.963109 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:36.963375 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:36.963991 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:36.964002 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:36.964500 (+ 498us) client_negotiation.cc:769] Sending connection context
0923 20:46:36.964850 (+ 350us) client_negotiation.cc:240] Negotiation successful
0923 20:46:36.965538 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":358,"threads_started":1}
I20260923 20:46:36.969023 9357 heartbeater.cc:347] Connected to a master server at 127.5.71.190:34321
I20260923 20:46:36.969380 9357 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:36.970182 9357 heartbeater.cc:511] Master 127.5.71.190:34321 requested a full tablet report, sending...
I20260923 20:46:36.971918 9034 ts_manager.cc:194] Registered new tserver with Master: f0e44653eb9c4c95872b25ee805397f7 (127.5.71.130:41693)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:36.973011 9034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:56897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:37.000656 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34321
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:37.338370 9363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:37.338721 9363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:37.338922 9363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:37.346786 9363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:37.346968 9363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:37.347150 9363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:37.357400 9363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:37.358929 9363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:37.360425 9363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:37.373931 9368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:37.373951 9371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:37.373931 9369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:37.375025 9363 server_base.cc:1034] running on GCE node
I20260923 20:46:37.375712 9363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:37.376724 9363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:37.377960 9363 hybrid_clock.cc:648] HybridClock initialized: now 1790196397377896 us; error 57 us; skew 500 ppm
I20260923 20:46:37.378286 9363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:37.381853 9363 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:37.383558 9363 webserver.cc:460] Webserver started at http://127.5.71.131:46813/ using document root <none> and password file <none>
I20260923 20:46:37.384336 9363 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:37.384477 9363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:37.384811 9363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:37.387833 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "a7ce176fccab41609db702fad8ba6d34"
format_stamp: "Formatted at 2026-09-23 20:46:37 on dist-test-slave-dn29"
server_key: "ee7b56ae6388d2214245a6e3d59d4df9"
server_key_iv: "4b8c9360fc6f15fbf5592d5f21bf3da2"
server_key_version: "encryptionkey@0"
I20260923 20:46:37.388662 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "a7ce176fccab41609db702fad8ba6d34"
format_stamp: "Formatted at 2026-09-23 20:46:37 on dist-test-slave-dn29"
server_key: "ee7b56ae6388d2214245a6e3d59d4df9"
server_key_iv: "4b8c9360fc6f15fbf5592d5f21bf3da2"
server_key_version: "encryptionkey@0"
I20260923 20:46:37.394313 9363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:37.398473 9378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:37.400189 9363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:37.400348 9363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "a7ce176fccab41609db702fad8ba6d34"
format_stamp: "Formatted at 2026-09-23 20:46:37 on dist-test-slave-dn29"
server_key: "ee7b56ae6388d2214245a6e3d59d4df9"
server_key_iv: "4b8c9360fc6f15fbf5592d5f21bf3da2"
server_key_version: "encryptionkey@0"
I20260923 20:46:37.400538 9363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:37.420089 9363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:37.424665 9363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:37.424961 9363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:37.425990 9363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:37.427381 9363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:37.427482 9363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:37.427623 9363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:37.427693 9363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:37.457005 9223 heartbeater.cc:503] Master 127.5.71.190:34321 was elected leader, sending a full tablet report...
I20260923 20:46:37.464982 9363 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:38243
I20260923 20:46:37.465016 9490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:38243 every 8 connection(s)
I20260923 20:46:37.466542 9363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:37.471172 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9363
I20260923 20:46:37.471405 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:37.472020 5406 external_mini_cluster.cc:1428] Setting key c4517c8449a2f80b686f8cc9ffb767d3
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:37.489570 9493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.469408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:47573 (local address 127.5.71.190:34321)
0923 20:46:37.469956 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:37.469962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:37.470979 (+ 1017us) server_negotiation.cc:407] Connection header received
0923 20:46:37.473524 (+ 2545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:37.473530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:37.473636 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:37.473754 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:37.475476 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.476270 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.477831 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.478227 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.478356 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.482535 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:37.482540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:37.482544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:37.482582 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:37.485660 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.486638 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.486644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.486648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.486814 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.487425 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.487430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.487434 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.487846 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:37.487999 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:37.489087 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0923 20:46:37.489278 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":299,"threads_started":1}
I20260923 20:46:37.490418 9494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.469941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:34321 (local address 127.5.71.131:47573)
0923 20:46:37.470749 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:37.470830 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:37.473210 (+ 2380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:37.474025 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:37.474038 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:37.474638 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:37.475317 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:37.475336 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.476419 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.476429 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:37.477679 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:37.477690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.478466 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.478474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:37.478710 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.479561 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:37.479593 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:37.482281 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:37.485843 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:37.485854 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:37.485876 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:37.486461 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:37.486978 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:37.486984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:37.486987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:37.487287 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:37.487988 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:37.487995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:37.488533 (+ 538us) client_negotiation.cc:769] Sending connection context
0923 20:46:37.488822 (+ 289us) client_negotiation.cc:240] Negotiation successful
0923 20:46:37.489255 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":307,"threads_started":1}
I20260923 20:46:37.492182 9491 heartbeater.cc:347] Connected to a master server at 127.5.71.190:34321
I20260923 20:46:37.492548 9491 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:37.493374 9491 heartbeater.cc:511] Master 127.5.71.190:34321 requested a full tablet report, sending...
I20260923 20:46:37.494993 9034 ts_manager.cc:194] Registered new tserver with Master: a7ce176fccab41609db702fad8ba6d34 (127.5.71.131:38243)
I20260923 20:46:37.495893 9034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:47573
I20260923 20:46:37.500965 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:37.564014 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.545583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50756 (local address 127.5.71.129:39029)
0923 20:46:37.548324 (+ 2741us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:37.548332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:37.548362 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:46:37.548424 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:37.548431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:37.548660 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:37.548824 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:37.550452 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.551501 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.552966 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.553614 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.553750 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.558038 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:37.558070 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:37.558087 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:37.558135 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:37.561075 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.561812 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.561820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.561823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.561953 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.562327 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.562333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.562337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.562691 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:37.563013 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:37.563325 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 20:46:37.563607 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2548,"thread_start_us":2395,"threads_started":1}
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:37.585923 9493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.570127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.5.71.190:34321)
0923 20:46:37.570373 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:37.570377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:37.570575 (+ 198us) server_negotiation.cc:407] Connection header received
0923 20:46:37.570713 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:37.570717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:37.570808 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:37.570905 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:37.572319 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.573146 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.574423 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.574871 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.574994 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.579707 (+ 4713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:37.579713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:37.579716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:37.579882 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:37.582930 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.583679 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.583686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.583690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.583890 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.584316 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.584322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.584326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.584681 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:37.584952 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:37.585311 (+ 359us) server_negotiation.cc:299] Negotiation successful
0923 20:46:37.585570 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
W20260923 20:46:37.586555 9034 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49508
I20260923 20:46:37.587746 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9095
I20260923 20:46:37.601178 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.588982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.5.71.129:39029)
0923 20:46:37.589184 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:37.589192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:37.589283 (+ 91us) server_negotiation.cc:407] Connection header received
0923 20:46:37.589493 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:37.589497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:37.589562 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:37.589662 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:37.590962 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.591684 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.592902 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.593313 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.593454 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.595847 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:37.595855 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:37.595860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:37.595901 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:37.598496 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.599138 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.599145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.599148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.599280 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.599646 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.599652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.599655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.600004 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:37.600296 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:37.600615 (+ 319us) server_negotiation.cc:299] Negotiation successful
0923 20:46:37.600846 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:46:37.601907 9218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:37.686956 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9095
I20260923 20:46:37.704533 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9229
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:37.721984 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.705524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.5.71.130:41693)
0923 20:46:37.706067 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:37.706074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:37.706102 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:46:37.706172 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:37.706179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:37.706417 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:37.706554 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:37.707762 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.709302 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.710572 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.711186 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.711292 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.715561 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:37.715573 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:37.715590 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:37.715626 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:37.718724 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.719568 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.719577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.719579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.719704 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.720278 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.720282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.720284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.720645 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:37.720911 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:37.721345 (+ 434us) server_negotiation.cc:299] Negotiation successful
0923 20:46:37.721591 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":290,"threads_started":1}
I20260923 20:46:37.723434 9352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:37.811595 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9229
I20260923 20:46:37.827235 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9363
Sep 23 20:46:37 dist-test-slave-dn29 krb5kdc[8989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:37.845463 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:37.828355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.5.71.131:38243)
0923 20:46:37.829022 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:37.829030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:37.829062 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:46:37.829130 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:37.829137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:37.829385 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:37.829539 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:37.830976 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.832344 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.833870 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:37.834525 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:37.834637 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:37.839571 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:37.839586 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:37.839605 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:37.839647 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:37.842676 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.843344 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.843352 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.843354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.843475 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:37.843860 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:37.843866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:37.843869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:37.844221 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:37.844521 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:37.844834 (+ 313us) server_negotiation.cc:299] Negotiation successful
0923 20:46:37.845102 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":354,"threads_started":1}
I20260923 20:46:37.846876 9486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:37.931571 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9363
I20260923 20:46:37.948107 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9002
I20260923 20:46:37.949415 9064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:38.054914 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9002
2026-09-23T20:46:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4085 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:38 dist-test-slave-dn29 krb5kdc[9511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:38 dist-test-slave-dn29 krb5kdc[9511](info): set up 2 sockets
Sep 23 20:46:38 dist-test-slave-dn29 krb5kdc[9511](info): commencing operation
krb5kdc: starting...
W20260923 20:46:39.188805 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:39 dist-test-slave-dn29 krb5kdc[9511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:39.404625 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42959
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:42959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:39.744280 9524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:39.744689 9524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:39.744844 9524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:39.752563 9524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:39.752758 9524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:39.752866 9524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:39.752964 9524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:39.753041 9524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:39.763706 9524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:42959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:39.765162 9524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:39.766601 9524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:39.779328 9529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:39.779287 9530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:39.779356 9532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:39.780423 9524 server_base.cc:1034] running on GCE node
I20260923 20:46:39.781250 9524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:39.782851 9524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:39.784051 9524 hybrid_clock.cc:648] HybridClock initialized: now 1790196399783995 us; error 49 us; skew 500 ppm
I20260923 20:46:39.784425 9524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:39 dist-test-slave-dn29 krb5kdc[9511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196399, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:39.787886 9524 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:39.789324 9524 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:39.789498 9524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:39.789865 9524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:39.793437 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "367b84bcd1d04168b709db425d3d65a7"
format_stamp: "Formatted at 2026-09-23 20:46:39 on dist-test-slave-dn29"
server_key: "1ae669033d91f0d2a63ef407bb940eb6"
server_key_iv: "972ba5d746ec52c1af9342b9ec341e03"
server_key_version: "encryptionkey@0"
I20260923 20:46:39.794376 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "367b84bcd1d04168b709db425d3d65a7"
format_stamp: "Formatted at 2026-09-23 20:46:39 on dist-test-slave-dn29"
server_key: "1ae669033d91f0d2a63ef407bb940eb6"
server_key_iv: "972ba5d746ec52c1af9342b9ec341e03"
server_key_version: "encryptionkey@0"
I20260923 20:46:39.800374 9524 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260923 20:46:39.804351 9538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:39.805976 9524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:46:39.806192 9524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "367b84bcd1d04168b709db425d3d65a7"
format_stamp: "Formatted at 2026-09-23 20:46:39 on dist-test-slave-dn29"
server_key: "1ae669033d91f0d2a63ef407bb940eb6"
server_key_iv: "972ba5d746ec52c1af9342b9ec341e03"
server_key_version: "encryptionkey@0"
I20260923 20:46:39.806433 9524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:39.825538 9524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:39.830646 9524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:39.830996 9524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:39.850616 9524 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:42959
I20260923 20:46:39.850618 9589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:42959 every 8 connection(s)
I20260923 20:46:39.851604 9524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:39.856015 9590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:39.856294 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9524
I20260923 20:46:39.856541 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:39.857388 5406 external_mini_cluster.cc:1428] Setting key 30cc432917bbdaf88c14de2d91be249c
I20260923 20:46:39.871025 9590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7: Bootstrap starting.
Sep 23 20:46:39 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:39.874850 9590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:39.876065 9590 log.cc:824] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:39.879657 9590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7: No bootstrap required, opened a new log
I20260923 20:46:39.882998 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:39.859555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.5.71.190:42959)
0923 20:46:39.860453 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:39.860464 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:39.860506 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:46:39.862507 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:39.862533 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:39.862963 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:39.863336 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:39.865216 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:39.866512 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:39.868449 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:39.869310 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:39.869477 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:39.874273 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:39.874282 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:39.874300 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:39.874342 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:39.878076 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:39.878872 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:39.878881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:39.878889 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:39.879096 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:39.879523 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:39.879527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:39.879530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:39.880315 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:39.880616 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:39.881077 (+ 461us) server_negotiation.cc:299] Negotiation successful
0923 20:46:39.881548 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":466,"threads_started":1}
I20260923 20:46:39.888578 9590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367b84bcd1d04168b709db425d3d65a7" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42959 } }
I20260923 20:46:39.888962 9590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:39.889034 9590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 367b84bcd1d04168b709db425d3d65a7, State: Initialized, Role: FOLLOWER
I20260923 20:46:39.889530 9590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367b84bcd1d04168b709db425d3d65a7" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42959 } }
I20260923 20:46:39.889737 9590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:39.889833 9590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:39.889964 9590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:39.892603 9590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367b84bcd1d04168b709db425d3d65a7" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42959 } }
I20260923 20:46:39.893213 9590 leader_election.cc:302] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 367b84bcd1d04168b709db425d3d65a7; no voters:
I20260923 20:46:39.893822 9590 leader_election.cc:288] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:39.893977 9595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:39.894726 9595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [term 1 LEADER]: Becoming Leader. State: Replica: 367b84bcd1d04168b709db425d3d65a7, State: Running, Role: LEADER
I20260923 20:46:39.895360 9595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367b84bcd1d04168b709db425d3d65a7" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42959 } }
I20260923 20:46:39.896199 9590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:39.898270 9596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "367b84bcd1d04168b709db425d3d65a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367b84bcd1d04168b709db425d3d65a7" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42959 } } }
I20260923 20:46:39.898497 9597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 367b84bcd1d04168b709db425d3d65a7. Latest consensus state: current_term: 1 leader_uuid: "367b84bcd1d04168b709db425d3d65a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367b84bcd1d04168b709db425d3d65a7" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42959 } } }
I20260923 20:46:39.898725 9596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:39.898797 9597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:39.902199 9602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:39.908169 9602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:39.917723 9602 catalog_manager.cc:1347] Generated new cluster ID: 329214669aa248efb5c4b88181e949c9
I20260923 20:46:39.917912 9602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:39.957432 9602 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:39.959441 9602 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:39.986552 9602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7: Generated new TSK 0
I20260923 20:46:39.987762 9602 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:40.044353 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42959
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:40.378152 9616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:40.378443 9616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:40.378597 9616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:40.386050 9616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:40.386160 9616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:40.386355 9616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:40.396626 9616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:40.398155 9616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:40.399560 9616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:40.413254 9621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:40.413251 9624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:40.413285 9622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:40.413986 9616 server_base.cc:1034] running on GCE node
I20260923 20:46:40.414633 9616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:40.415567 9616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:40.416764 9616 hybrid_clock.cc:648] HybridClock initialized: now 1790196400416701 us; error 51 us; skew 500 ppm
I20260923 20:46:40.417150 9616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:40 dist-test-slave-dn29 krb5kdc[9511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:40.420433 9616 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:40.421572 9616 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:40.421660 9616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:40.421924 9616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:40.424940 9616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "c7f9f1c69f124163a4e78f8842d9b08f"
format_stamp: "Formatted at 2026-09-23 20:46:40 on dist-test-slave-dn29"
server_key: "a2e5c6342ebe923a8caf1515eb0612ad"
server_key_iv: "2a087776baa8da7fed3c7d93f392658a"
server_key_version: "encryptionkey@0"
I20260923 20:46:40.425724 9616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "c7f9f1c69f124163a4e78f8842d9b08f"
format_stamp: "Formatted at 2026-09-23 20:46:40 on dist-test-slave-dn29"
server_key: "a2e5c6342ebe923a8caf1515eb0612ad"
server_key_iv: "2a087776baa8da7fed3c7d93f392658a"
server_key_version: "encryptionkey@0"
I20260923 20:46:40.431025 9616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:40.434986 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:40.436586 9616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:40.436756 9616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "c7f9f1c69f124163a4e78f8842d9b08f"
format_stamp: "Formatted at 2026-09-23 20:46:40 on dist-test-slave-dn29"
server_key: "a2e5c6342ebe923a8caf1515eb0612ad"
server_key_iv: "2a087776baa8da7fed3c7d93f392658a"
server_key_version: "encryptionkey@0"
I20260923 20:46:40.437040 9616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:40.464884 9616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:40.469472 9616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:40.469785 9616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:40.470883 9616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:40.472389 9616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:40.472477 9616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:40.472616 9616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:40.472694 9616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:40.507695 9616 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:34299
I20260923 20:46:40.507838 9742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:34299 every 8 connection(s)
I20260923 20:46:40.508605 9616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:40.515265 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9616
I20260923 20:46:40.515512 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:40.516371 5406 external_mini_cluster.cc:1428] Setting key 88cfec1e0494b810a6853f3fc12c3887
Sep 23 20:46:40 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:40.535486 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:40.511273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:33997 (local address 127.5.71.190:42959)
0923 20:46:40.511923 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:40.511942 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:40.517371 (+ 5429us) server_negotiation.cc:407] Connection header received
0923 20:46:40.519891 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:40.519897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:40.519972 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:40.520064 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:40.522413 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:40.523145 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:40.525579 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:40.526002 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:40.526142 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:40.529689 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:40.529699 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:40.529703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:40.529735 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:40.532427 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:40.533169 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:40.533179 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:40.533181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:40.533298 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:40.533717 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:40.533722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:40.533726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:40.533993 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:40.534147 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:40.534882 (+ 735us) server_negotiation.cc:299] Negotiation successful
0923 20:46:40.535109 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":263,"threads_started":1}
I20260923 20:46:40.536592 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:40.512056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42959 (local address 127.5.71.129:33997)
0923 20:46:40.517145 (+ 5089us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:40.517201 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:40.519491 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:40.520391 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:40.520408 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:40.521304 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:40.522214 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:40.522244 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:40.523315 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:40.523324 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:40.525411 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:40.525430 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:40.526245 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:40.526251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:40.526496 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:40.527222 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:40.527246 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:40.529454 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:40.532574 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:40.532584 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:40.532602 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:40.533039 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:40.533404 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:40.533409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:40.533412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:40.533601 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:40.534119 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:40.534127 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:40.534498 (+ 371us) client_negotiation.cc:769] Sending connection context
0923 20:46:40.534787 (+ 289us) client_negotiation.cc:240] Negotiation successful
0923 20:46:40.535335 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2523,"thread_start_us":2292,"threads_started":1}
I20260923 20:46:40.538314 9743 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:40.538642 9743 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:40.539333 9743 heartbeater.cc:511] Master 127.5.71.190:42959 requested a full tablet report, sending...
I20260923 20:46:40.541744 9555 ts_manager.cc:194] Registered new tserver with Master: c7f9f1c69f124163a4e78f8842d9b08f (127.5.71.129:34299)
I20260923 20:46:40.543211 9555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:33997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:40.564155 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42959
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:40.896111 9749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:40.896425 9749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:40.896523 9749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:40.904295 9749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:40.904417 9749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:40.904597 9749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:40.914947 9749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:40.916325 9749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:40.917694 9749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:40.931270 9754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:40.931252 9755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:40.931393 9757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:40.932384 9749 server_base.cc:1034] running on GCE node
I20260923 20:46:40.933118 9749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:40.934221 9749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:40.935467 9749 hybrid_clock.cc:648] HybridClock initialized: now 1790196400935416 us; error 32 us; skew 500 ppm
I20260923 20:46:40.936035 9749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:40 dist-test-slave-dn29 krb5kdc[9511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:40.947300 9749 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:40.948642 9749 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:40.948750 9749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:40.949153 9749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:40.952589 9749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "5ef0e182bdb644d3882f50f93484efe9"
format_stamp: "Formatted at 2026-09-23 20:46:40 on dist-test-slave-dn29"
server_key: "f93a05d41f2608c8b2c32eb6192cbdf7"
server_key_iv: "13c8f5fdd2721327777d87da54e79fcb"
server_key_version: "encryptionkey@0"
I20260923 20:46:40.953449 9749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "5ef0e182bdb644d3882f50f93484efe9"
format_stamp: "Formatted at 2026-09-23 20:46:40 on dist-test-slave-dn29"
server_key: "f93a05d41f2608c8b2c32eb6192cbdf7"
server_key_iv: "13c8f5fdd2721327777d87da54e79fcb"
server_key_version: "encryptionkey@0"
I20260923 20:46:40.959072 9749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:40.963182 9763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:40.964839 9749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:40.965055 9749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "5ef0e182bdb644d3882f50f93484efe9"
format_stamp: "Formatted at 2026-09-23 20:46:40 on dist-test-slave-dn29"
server_key: "f93a05d41f2608c8b2c32eb6192cbdf7"
server_key_iv: "13c8f5fdd2721327777d87da54e79fcb"
server_key_version: "encryptionkey@0"
I20260923 20:46:40.965323 9749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:40.987699 9749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:40.992370 9749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:40.992654 9749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:40.993754 9749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:40.995602 9749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:40.995723 9749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:40.995947 9749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:40.996033 9749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:41.033704 9749 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:39519
I20260923 20:46:41.033893 9875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:39519 every 8 connection(s)
I20260923 20:46:41.034637 9749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:41.035393 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9749
I20260923 20:46:41.035596 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:41.036275 5406 external_mini_cluster.cc:1428] Setting key d3102ffe350c22e298e9049c330697dd
Sep 23 20:46:41 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:41.059545 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.037616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:55247 (local address 127.5.71.190:42959)
0923 20:46:41.038097 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.038104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.039259 (+ 1155us) server_negotiation.cc:407] Connection header received
0923 20:46:41.041524 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.041529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.041629 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.041756 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:41.043505 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.044761 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.046796 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.047658 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.047844 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.052005 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:41.052016 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:41.052026 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:41.052084 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:41.056050 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:41.056903 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:41.056913 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:41.056919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:41.057094 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:41.057572 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:41.057577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:41.057579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:41.058112 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:41.058250 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.059019 (+ 769us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.059270 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:41.060911 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.038205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42959 (local address 127.5.71.130:55247)
0923 20:46:41.039081 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.039135 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.041233 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.042032 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.042041 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.042659 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:41.043306 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.043325 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.044952 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.044956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.046631 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.046644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.047967 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.047974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.048169 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.049029 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:41.049068 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:41.051699 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:41.056233 (+ 4534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:41.056242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:41.056257 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:41.056744 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:41.057247 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:41.057251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:41.057253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:41.057438 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:41.058271 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:41.058278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:41.058619 (+ 341us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.058882 (+ 263us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.059405 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":393,"threads_started":1}
I20260923 20:46:41.062740 9876 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42959
I20260923 20:46:41.063115 9876 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:41.063968 9876 heartbeater.cc:511] Master 127.5.71.190:42959 requested a full tablet report, sending...
I20260923 20:46:41.065554 9555 ts_manager.cc:194] Registered new tserver with Master: 5ef0e182bdb644d3882f50f93484efe9 (127.5.71.130:39519)
I20260923 20:46:41.066860 9555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:55247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:41.087744 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42959
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:41.431113 9882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:41.431425 9882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:41.431594 9882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:41.438956 9882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:41.439077 9882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:41.439316 9882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:41.449625 9882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42959
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:41.451188 9882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:41.452666 9882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:41.466198 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:41.466143 9888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:41.466112 9890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:41.467074 9882 server_base.cc:1034] running on GCE node
I20260923 20:46:41.467732 9882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:41.468811 9882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:41.470036 9882 hybrid_clock.cc:648] HybridClock initialized: now 1790196401469982 us; error 46 us; skew 500 ppm
I20260923 20:46:41.470461 9882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:41 dist-test-slave-dn29 krb5kdc[9511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196401, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:41.473713 9882 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:41.474898 9882 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:41.474992 9882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:41.475360 9882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:41.478586 9882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "e3641eee9b034394850ff87ff864658a"
format_stamp: "Formatted at 2026-09-23 20:46:41 on dist-test-slave-dn29"
server_key: "347d298401a307c1a659a1c91817123b"
server_key_iv: "2f3300d2081cd3022488b65b59ffc65f"
server_key_version: "encryptionkey@0"
I20260923 20:46:41.479451 9882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "e3641eee9b034394850ff87ff864658a"
format_stamp: "Formatted at 2026-09-23 20:46:41 on dist-test-slave-dn29"
server_key: "347d298401a307c1a659a1c91817123b"
server_key_iv: "2f3300d2081cd3022488b65b59ffc65f"
server_key_version: "encryptionkey@0"
I20260923 20:46:41.484946 9882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260923 20:46:41.488910 9896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:41.490541 9882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:46:41.490759 9882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "e3641eee9b034394850ff87ff864658a"
format_stamp: "Formatted at 2026-09-23 20:46:41 on dist-test-slave-dn29"
server_key: "347d298401a307c1a659a1c91817123b"
server_key_iv: "2f3300d2081cd3022488b65b59ffc65f"
server_key_version: "encryptionkey@0"
I20260923 20:46:41.491009 9882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:41.546551 9743 heartbeater.cc:503] Master 127.5.71.190:42959 was elected leader, sending a full tablet report...
I20260923 20:46:41.554113 9882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:41.558660 9882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:41.558954 9882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:41.560146 9882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:41.561574 9882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:41.561700 9882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:41.561841 9882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:41.561929 9882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:41.597198 9882 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:38381
I20260923 20:46:41.597298 10008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:38381 every 8 connection(s)
I20260923 20:46:41.598228 9882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:41.599041 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 9882
I20260923 20:46:41.599272 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:41.599822 5406 external_mini_cluster.cc:1428] Setting key 1e5703ae2b892deb8c738be3323d3811
Sep 23 20:46:41 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196401, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:41.619616 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.601346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:56067 (local address 127.5.71.190:42959)
0923 20:46:41.601799 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.601807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.603040 (+ 1233us) server_negotiation.cc:407] Connection header received
0923 20:46:41.605051 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.605056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.605124 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.605220 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:41.606904 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.607655 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.609459 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.609905 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.610024 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.613833 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:41.613844 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:41.613847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:41.613882 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:41.616554 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:41.617305 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:41.617312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:41.617315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:41.617434 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:41.617862 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:41.617867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:41.617870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:41.618157 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:41.618313 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.618999 (+ 686us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.619227 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":241,"threads_started":1}
I20260923 20:46:41.620724 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.601980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42959 (local address 127.5.71.131:56067)
0923 20:46:41.602895 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.602943 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.604788 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.605488 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.605497 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.606126 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:41.606726 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.606744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.608020 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.608024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.609317 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.609330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.610180 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.610186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.610396 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.611244 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:41.611273 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:41.613597 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:41.616698 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:41.616708 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:41.616725 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:41.617177 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:41.617541 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:41.617546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:41.617550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:41.617745 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:41.618283 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:41.618291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:41.618655 (+ 364us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.618918 (+ 263us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.619463 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":355,"threads_started":1}
I20260923 20:46:41.622160 10009 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42959
I20260923 20:46:41.622495 10009 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:41.623270 10009 heartbeater.cc:511] Master 127.5.71.190:42959 requested a full tablet report, sending...
I20260923 20:46:41.624910 9555 ts_manager.cc:194] Registered new tserver with Master: e3641eee9b034394850ff87ff864658a (127.5.71.131:38381)
I20260923 20:46:41.627221 9555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:56067
I20260923 20:46:41.628350 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:46:41.652547 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.634975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49368 (local address 127.5.71.190:42959)
0923 20:46:41.635177 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.635184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.636058 (+ 874us) server_negotiation.cc:407] Connection header received
0923 20:46:41.636349 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.636357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.636477 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.636600 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:41.639233 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.640255 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.641690 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.642416 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.642560 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.645065 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:41.645070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:41.645074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:41.645242 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:41.649086 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:41.649993 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:41.649998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:41.650002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:41.650147 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:41.650647 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:41.650652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:41.650655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:41.651190 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:41.651452 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.651900 (+ 448us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.652175 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 20:46:41.658146 9554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:41.666218 9554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:41.705694 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.680637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38381 (local address 127.0.0.1:44138)
0923 20:46:41.692422 (+ 11785us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.692501 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.692768 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.695445 (+ 2677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.695477 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.695517 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:41.696068 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.696085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.698742 (+ 2657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.698751 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.702594 (+ 3843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.702618 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.704626 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.704636 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.704923 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.704998 (+ 75us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.705140 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.705367 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11372,"thread_start_us":14562,"threads_started":1}
I20260923 20:46:41.705773 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.680837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.5.71.131:38381)
0923 20:46:41.694584 (+ 13747us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.694599 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.694648 (+ 49us) server_negotiation.cc:407] Connection header received
0923 20:46:41.694790 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.694801 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.695170 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.695388 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:41.696498 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.698559 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.702811 (+ 4252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.704403 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.704535 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.704783 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.705226 (+ 443us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.705411 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13487,"thread_start_us":13307,"threads_started":1}
I20260923 20:46:41.707768 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.681495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46280)
0923 20:46:41.695868 (+ 14373us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.695894 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.696052 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.700830 (+ 4778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.700835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.700860 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:41.701378 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.701388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.702846 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.702852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.704999 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.705010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.706955 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.706968 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.707130 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.707154 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.707223 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.707371 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14182,"mutex_wait_us":2,"thread_start_us":11522,"threads_started":1}
I20260923 20:46:41.708813 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.681736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.5.71.129:34299)
0923 20:46:41.700336 (+ 18600us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.700342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.700365 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:46:41.700443 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.700449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.700636 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.700848 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:41.701548 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.702680 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.705213 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.706708 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.708027 (+ 1319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.708279 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.708350 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.708522 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18435,"thread_start_us":18285,"threads_started":1}
I20260923 20:46:41.710515 9944 tablet_service.cc:1467] Processing CreateTablet for tablet 7b6fb0d908294fb8907c7067eb0ab856 (DEFAULT_TABLE table=test-table [id=7e640df9dd69444e9ac9bb871b694f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:41.712152 9944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b6fb0d908294fb8907c7067eb0ab856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:41.712697 9678 tablet_service.cc:1467] Processing CreateTablet for tablet 7b6fb0d908294fb8907c7067eb0ab856 (DEFAULT_TABLE table=test-table [id=7e640df9dd69444e9ac9bb871b694f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:41.713804 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.695632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:39519 (local address 127.0.0.1:58302)
0923 20:46:41.705047 (+ 9415us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.705066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.705191 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.706720 (+ 1529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.706726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.706754 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:41.707224 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.707240 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.709089 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.709095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.711397 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.711413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.713026 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.713033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.713231 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.713270 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.713382 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.713512 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9267,"mutex_wait_us":93,"thread_start_us":9548,"threads_started":1}
I20260923 20:46:41.714112 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b6fb0d908294fb8907c7067eb0ab856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:41.714531 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.695944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58302 (local address 127.5.71.130:39519)
0923 20:46:41.696871 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.696883 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.706084 (+ 9201us) server_negotiation.cc:407] Connection header received
0923 20:46:41.706201 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.706209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.706499 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.706725 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:41.707398 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.708754 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.711604 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.712808 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.712977 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.713202 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.714074 (+ 872us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.714281 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":496,"threads_started":1}
I20260923 20:46:41.717804 9811 tablet_service.cc:1467] Processing CreateTablet for tablet 7b6fb0d908294fb8907c7067eb0ab856 (DEFAULT_TABLE table=test-table [id=7e640df9dd69444e9ac9bb871b694f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:41.719020 9811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b6fb0d908294fb8907c7067eb0ab856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:41.728459 10029 tablet_bootstrap.cc:492] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Bootstrap starting.
I20260923 20:46:41.729350 10028 tablet_bootstrap.cc:492] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Bootstrap starting.
I20260923 20:46:41.733174 10030 tablet_bootstrap.cc:492] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Bootstrap starting.
I20260923 20:46:41.734730 10029 tablet_bootstrap.cc:654] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:41.735893 10029 log.cc:824] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:41.736223 10030 tablet_bootstrap.cc:654] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:41.737336 10030 log.cc:824] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:41.738550 10028 tablet_bootstrap.cc:654] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:41.739750 10029 tablet_bootstrap.cc:492] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: No bootstrap required, opened a new log
I20260923 20:46:41.739897 10028 log.cc:824] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:41.740103 10029 ts_tablet_manager.cc:1397] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260923 20:46:41.740688 10030 tablet_bootstrap.cc:492] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: No bootstrap required, opened a new log
I20260923 20:46:41.740973 10030 ts_tablet_manager.cc:1397] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260923 20:46:41.743126 10028 tablet_bootstrap.cc:492] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: No bootstrap required, opened a new log
I20260923 20:46:41.743417 10028 ts_tablet_manager.cc:1397] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260923 20:46:41.748078 10029 raft_consensus.cc:348] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.748478 10029 raft_consensus.cc:374] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:41.748570 10029 raft_consensus.cc:729] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3641eee9b034394850ff87ff864658a, State: Initialized, Role: FOLLOWER
I20260923 20:46:41.749168 10029 consensus_queue.cc:260] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.750313 10009 heartbeater.cc:503] Master 127.5.71.190:42959 was elected leader, sending a full tablet report...
I20260923 20:46:41.750928 10029 ts_tablet_manager.cc:1428] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260923 20:46:41.752511 10030 raft_consensus.cc:348] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.752921 10030 raft_consensus.cc:374] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:41.753046 10030 raft_consensus.cc:729] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef0e182bdb644d3882f50f93484efe9, State: Initialized, Role: FOLLOWER
I20260923 20:46:41.753669 10030 consensus_queue.cc:260] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.754998 9876 heartbeater.cc:503] Master 127.5.71.190:42959 was elected leader, sending a full tablet report...
I20260923 20:46:41.754980 10028 raft_consensus.cc:348] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.755405 10028 raft_consensus.cc:374] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:41.755491 10028 raft_consensus.cc:729] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f9f1c69f124163a4e78f8842d9b08f, State: Initialized, Role: FOLLOWER
I20260923 20:46:41.755564 10030 ts_tablet_manager.cc:1428] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260923 20:46:41.756140 10028 consensus_queue.cc:260] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.758939 10028 ts_tablet_manager.cc:1428] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260923 20:46:41.761024 9744 tablet.cc:2369] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:41.787078 9877 tablet.cc:2369] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:46:41.850178 10010 tablet.cc:2369] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:41.921531 10035 raft_consensus.cc:482] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:41.921871 10035 raft_consensus.cc:504] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.923748 10035 leader_election.cc:288] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3641eee9b034394850ff87ff864658a (127.5.71.131:38381), c7f9f1c69f124163a4e78f8842d9b08f (127.5.71.129:34299)
I20260923 20:46:41.930477 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.924351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:47405 (local address 127.5.71.129:34299)
0923 20:46:41.924596 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.924605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.925136 (+ 531us) server_negotiation.cc:407] Connection header received
0923 20:46:41.925366 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.925374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.925450 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.925549 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:41.926127 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.926857 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.928760 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.929533 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.929729 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.929803 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.930066 (+ 263us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.930208 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:46:41.930662 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.924177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:35433 (local address 127.5.71.131:38381)
0923 20:46:41.924422 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.924432 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.925300 (+ 868us) server_negotiation.cc:407] Connection header received
0923 20:46:41.925490 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.925494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.925554 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.925674 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:41.926264 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.927142 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.928951 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.929750 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.929946 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.930027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.930285 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.930411 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:46:41.930792 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.924222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.5.71.130:47405)
0923 20:46:41.925013 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.925030 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.925217 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.925566 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.925570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.925595 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:41.925973 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.925982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.926986 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.926993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.928610 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.928621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.929746 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.929752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.929908 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.929931 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.930044 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.930208 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":30,"thread_start_us":386,"threads_started":1}
I20260923 20:46:41.930820 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.924241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38381 (local address 127.5.71.130:35433)
0923 20:46:41.925192 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:41.925206 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:41.925358 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:41.925695 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:41.925704 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:41.925744 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:41.926120 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.926128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.927284 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.927291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.928803 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:41.928814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.929961 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.929967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:41.930137 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.930162 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 20:46:41.930260 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 20:46:41.930386 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":32,"thread_start_us":652,"threads_started":1}
I20260923 20:46:41.932124 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6fb0d908294fb8907c7067eb0ab856" candidate_uuid: "5ef0e182bdb644d3882f50f93484efe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" is_pre_election: true
I20260923 20:46:41.932227 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6fb0d908294fb8907c7067eb0ab856" candidate_uuid: "5ef0e182bdb644d3882f50f93484efe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3641eee9b034394850ff87ff864658a" is_pre_election: true
I20260923 20:46:41.932662 9698 raft_consensus.cc:2454] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ef0e182bdb644d3882f50f93484efe9 in term 0.
I20260923 20:46:41.932708 9964 raft_consensus.cc:2454] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ef0e182bdb644d3882f50f93484efe9 in term 0.
I20260923 20:46:41.933351 9764 leader_election.cc:302] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ef0e182bdb644d3882f50f93484efe9, e3641eee9b034394850ff87ff864658a; no voters:
I20260923 20:46:41.933872 10035 raft_consensus.cc:2781] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:41.934015 10035 raft_consensus.cc:482] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:41.934100 10035 raft_consensus.cc:3037] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:41.936275 10035 raft_consensus.cc:504] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.936863 10035 leader_election.cc:288] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [CANDIDATE]: Term 1 election: Requested vote from peers e3641eee9b034394850ff87ff864658a (127.5.71.131:38381), c7f9f1c69f124163a4e78f8842d9b08f (127.5.71.129:34299)
I20260923 20:46:41.937400 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6fb0d908294fb8907c7067eb0ab856" candidate_uuid: "5ef0e182bdb644d3882f50f93484efe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f9f1c69f124163a4e78f8842d9b08f"
I20260923 20:46:41.937440 9964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6fb0d908294fb8907c7067eb0ab856" candidate_uuid: "5ef0e182bdb644d3882f50f93484efe9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3641eee9b034394850ff87ff864658a"
I20260923 20:46:41.937608 9698 raft_consensus.cc:3037] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:41.937629 9964 raft_consensus.cc:3037] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:41.939920 9964 raft_consensus.cc:2454] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ef0e182bdb644d3882f50f93484efe9 in term 1.
I20260923 20:46:41.939955 9698 raft_consensus.cc:2454] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ef0e182bdb644d3882f50f93484efe9 in term 1.
I20260923 20:46:41.940409 9764 leader_election.cc:302] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ef0e182bdb644d3882f50f93484efe9, e3641eee9b034394850ff87ff864658a; no voters:
I20260923 20:46:41.940768 10035 raft_consensus.cc:2781] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:41.941244 10035 raft_consensus.cc:686] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 1 LEADER]: Becoming Leader. State: Replica: 5ef0e182bdb644d3882f50f93484efe9, State: Running, Role: LEADER
I20260923 20:46:41.941845 10035 consensus_queue.cc:237] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } }
I20260923 20:46:41.946539 9554 catalog_manager.cc:5495] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ef0e182bdb644d3882f50f93484efe9 (127.5.71.130). New cstate: current_term: 1 leader_uuid: "5ef0e182bdb644d3882f50f93484efe9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ef0e182bdb644d3882f50f93484efe9" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 } health_report { overall_health: UNKNOWN } } }
I20260923 20:46:41.990414 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:41.984262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.5.71.130:39519)
0923 20:46:41.984512 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:41.984518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:41.984540 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:46:41.984844 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:41.984849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:41.984954 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:41.985137 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:46:41.985557 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.986377 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.988005 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:41.988521 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:41.988695 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:41.989213 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:46:41.989786 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:46:41.989960 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:41.990021 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 20:46:41.990110 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 20:46:41.999874 9698 raft_consensus.cc:1261] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 1 FOLLOWER]: Refusing update from remote peer 5ef0e182bdb644d3882f50f93484efe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:41.999889 9964 raft_consensus.cc:1261] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 1 FOLLOWER]: Refusing update from remote peer 5ef0e182bdb644d3882f50f93484efe9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:42.000912 10039 consensus_queue.cc:1035] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f9f1c69f124163a4e78f8842d9b08f" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 34299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:42.001333 10035 consensus_queue.cc:1035] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3641eee9b034394850ff87ff864658a" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:42.019430 10044 mvcc.cc:205] Tried to move back new op lower bound from 7332644462580371456 to 7332644462357553152. Current Snapshot: MvccSnapshot[applied={T|T < 7332644462580371456}]
I20260923 20:46:42.033380 9554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:42.033661 9554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:42.037371 9554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 367b84bcd1d04168b709db425d3d65a7: Sending DeleteTablet for 3 replicas of tablet 7b6fb0d908294fb8907c7067eb0ab856
I20260923 20:46:42.039017 9678 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b6fb0d908294fb8907c7067eb0ab856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:42 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:46280
I20260923 20:46:42.039374 9944 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b6fb0d908294fb8907c7067eb0ab856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:42 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:44138
I20260923 20:46:42.040501 9811 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b6fb0d908294fb8907c7067eb0ab856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:42 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:58302
I20260923 20:46:42.043195 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9616
I20260923 20:46:42.064911 10054 tablet_replica.cc:330] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: stopping tablet replica
I20260923 20:46:42.065815 10053 tablet_replica.cc:330] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: stopping tablet replica
I20260923 20:46:42.066613 10054 raft_consensus.cc:2229] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:42.067335 10053 raft_consensus.cc:2229] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:42.068400 10053 raft_consensus.cc:2258] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:42.071388 10054 raft_consensus.cc:2258] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:42.072495 10053 ts_tablet_manager.cc:1905] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:42.073868 10054 ts_tablet_manager.cc:1905] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 20:46:42 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:42.079773 10053 ts_tablet_manager.cc:1918] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:42.080013 10053 log.cc:1196] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/7b6fb0d908294fb8907c7067eb0ab856
I20260923 20:46:42.080564 10053 ts_tablet_manager.cc:1939] T 7b6fb0d908294fb8907c7067eb0ab856 P e3641eee9b034394850ff87ff864658a: Deleting consensus metadata
I20260923 20:46:42.082266 10054 ts_tablet_manager.cc:1918] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:42.082557 10054 log.cc:1196] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/7b6fb0d908294fb8907c7067eb0ab856
I20260923 20:46:42.082695 9539 catalog_manager.cc:4841] TS e3641eee9b034394850ff87ff864658a (127.5.71.131:38381): tablet 7b6fb0d908294fb8907c7067eb0ab856 (table test-table [id=7e640df9dd69444e9ac9bb871b694f5e]) successfully deleted
I20260923 20:46:42.083279 10052 tablet_replica.cc:330] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: stopping tablet replica
I20260923 20:46:42.083316 10054 ts_tablet_manager.cc:1939] T 7b6fb0d908294fb8907c7067eb0ab856 P 5ef0e182bdb644d3882f50f93484efe9: Deleting consensus metadata
I20260923 20:46:42.083851 10052 raft_consensus.cc:2229] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:42.084462 10052 raft_consensus.cc:2258] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:42.085541 9539 catalog_manager.cc:4841] TS 5ef0e182bdb644d3882f50f93484efe9 (127.5.71.130:39519): tablet 7b6fb0d908294fb8907c7067eb0ab856 (table test-table [id=7e640df9dd69444e9ac9bb871b694f5e]) successfully deleted
I20260923 20:46:42.086990 10052 ts_tablet_manager.cc:1905] T 7b6fb0d908294fb8907c7067eb0ab856 P c7f9f1c69f124163a4e78f8842d9b08f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:42.089956 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.045600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.5.71.129:34299)
0923 20:46:42.045811 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:42.045818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:42.062653 (+ 16835us) server_negotiation.cc:407] Connection header received
0923 20:46:42.063202 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:42.063209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:42.063307 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:42.063404 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:42.065534 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:42.066825 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:42.068956 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:42.069671 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:42.069788 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:42.081256 (+ 11468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:42.081277 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:42.081297 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:42.081366 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:42.085652 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:42.086712 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:42.086724 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:42.086727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:42.086940 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:42.087650 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:42.087656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:42.087660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:42.088218 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:42.088542 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:42.089197 (+ 655us) server_negotiation.cc:299] Negotiation successful
0923 20:46:42.089468 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:46:42.092078 9738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:42.205775 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9616
W20260923 20:46:42.223682 9541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:34299 (error 108)
I20260923 20:46:42.224174 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9749
I20260923 20:46:42.226207 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.225397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46306)
0923 20:46:42.225839 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.225985 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":232,"threads_started":1}
W20260923 20:46:42.226483 9541 catalog_manager.cc:4584] TS c7f9f1c69f124163a4e78f8842d9b08f (127.5.71.129:34299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b6fb0d908294fb8907c7067eb0ab856: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Sep 23 20:46:42 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:42.242647 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.225004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.5.71.130:39519)
0923 20:46:42.225171 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:42.225176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:42.225309 (+ 133us) server_negotiation.cc:407] Connection header received
0923 20:46:42.225688 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:42.225694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:42.225804 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:42.225911 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:42.227363 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:42.228620 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:42.229921 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:42.230606 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:42.230740 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:42.235280 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:42.235300 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:42.235326 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:42.235390 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:42.239513 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:42.240306 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:42.240316 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:42.240320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:42.240495 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:42.240916 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:42.240921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:42.240926 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:42.241400 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:42.241730 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:42.241972 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 20:46:42.242200 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 20:46:42.243541 9871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:42.280253 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.279661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46310)
0923 20:46:42.279976 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.280092 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260923 20:46:42.281364 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.280940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46326)
0923 20:46:42.281150 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.281222 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260923 20:46:42.342046 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.341562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46336)
0923 20:46:42.341788 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.341888 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260923 20:46:42.343086 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.342706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46338)
0923 20:46:42.342855 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.342947 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260923 20:46:42.360702 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9749
I20260923 20:46:42.378718 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9882
Sep 23 20:46:42 dist-test-slave-dn29 krb5kdc[9511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:42.393909 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.379471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.5.71.131:38381)
0923 20:46:42.379773 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:42.379781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:42.379838 (+ 57us) server_negotiation.cc:407] Connection header received
0923 20:46:42.379906 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:42.379910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:42.379972 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:42.380090 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:42.381383 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:42.382101 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:42.383386 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:42.383867 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:42.383992 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:42.388470 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:42.388486 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:42.388506 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:42.388546 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:42.391241 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:42.391992 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:42.392000 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:42.392002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:42.392159 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:42.392503 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:42.392507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:42.392509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:42.392884 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:42.393137 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:42.393367 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 20:46:42.393530 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 20:46:42.394649 10004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:42.423899 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.423322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46340)
0923 20:46:42.423562 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.423668 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260923 20:46:42.424932 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.424521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46344)
0923 20:46:42.424729 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.424797 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260923 20:46:42.485189 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9882
I20260923 20:46:42.501881 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 9524
I20260923 20:46:42.503144 9585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:42.618338 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:42.617102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:34299 (local address 127.0.0.1:46346)
0923 20:46:42.617581 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:42.617927 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260923 20:46:42.618305 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 9524
2026-09-23T20:46:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4565 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T20:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:42Z Disabled control of system clock
I20260923 20:46:42.689776 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:44289
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:39651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:44289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 20:46:43.027150 10065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:43.027462 10065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:43.027609 10065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:43.035032 10065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:43.035156 10065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:43.035230 10065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:43.035321 10065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:43.035382 10065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:43.045217 10065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:44289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:44289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:43.046583 10065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:43.048687 10065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:43.061409 10070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:43.061410 10071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:43.061642 10073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:43.062650 10065 server_base.cc:1034] running on GCE node
I20260923 20:46:43.063474 10065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:43.065060 10065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:43.066284 10065 hybrid_clock.cc:648] HybridClock initialized: now 1790196403066246 us; error 36 us; skew 500 ppm
I20260923 20:46:43.066629 10065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:43.069181 10065 webserver.cc:460] Webserver started at http://127.5.71.190:41349/ using document root <none> and password file <none>
I20260923 20:46:43.069887 10065 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:43.069977 10065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:43.070242 10065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:43.073287 10065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "ed924fcf23344bfca8c4e03a2c87726e"
format_stamp: "Formatted at 2026-09-23 20:46:43 on dist-test-slave-dn29"
server_key: "570955a622e224c3c0bc606812775314"
server_key_iv: "795728851d5566132768e8333c45540b"
server_key_version: "encryptionkey@0"
I20260923 20:46:43.074092 10065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "ed924fcf23344bfca8c4e03a2c87726e"
format_stamp: "Formatted at 2026-09-23 20:46:43 on dist-test-slave-dn29"
server_key: "570955a622e224c3c0bc606812775314"
server_key_iv: "795728851d5566132768e8333c45540b"
server_key_version: "encryptionkey@0"
I20260923 20:46:43.079514 10065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:43.083154 10079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:43.084635 10065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:43.084837 10065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "ed924fcf23344bfca8c4e03a2c87726e"
format_stamp: "Formatted at 2026-09-23 20:46:43 on dist-test-slave-dn29"
server_key: "570955a622e224c3c0bc606812775314"
server_key_iv: "795728851d5566132768e8333c45540b"
server_key_version: "encryptionkey@0"
I20260923 20:46:43.085064 10065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:43.091296 10065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:43.092164 10065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:43.092465 10065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:43.111899 10065 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:44289
I20260923 20:46:43.111958 10130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:44289 every 8 connection(s)
I20260923 20:46:43.113293 10065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:43.117497 10131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:43.120270 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10065
I20260923 20:46:43.120473 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:43.121376 5406 external_mini_cluster.cc:1428] Setting key 7d237f8c08c80ee9ea964a42385d793e
I20260923 20:46:43.130330 10134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:43.123719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49704 (local address 127.5.71.190:44289)
0923 20:46:43.124879 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:43.124890 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:43.124975 (+ 85us) server_negotiation.cc:407] Connection header received
0923 20:46:43.126696 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:43.126723 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:43.127157 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:43.127549 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:43.128658 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:43.128664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:43.128677 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:43.128691 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:43.128752 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:43.128771 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:43.129033 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:43.129187 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 20:46:43.129454 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":364,"threads_started":1}
I20260923 20:46:43.131392 10131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e: Bootstrap starting.
I20260923 20:46:43.134855 10131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:43.136024 10131 log.cc:824] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:43.139292 10131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e: No bootstrap required, opened a new log
I20260923 20:46:43.147891 10131 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed924fcf23344bfca8c4e03a2c87726e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44289 } }
I20260923 20:46:43.148329 10131 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:43.148474 10131 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed924fcf23344bfca8c4e03a2c87726e, State: Initialized, Role: FOLLOWER
I20260923 20:46:43.149111 10131 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed924fcf23344bfca8c4e03a2c87726e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44289 } }
I20260923 20:46:43.149379 10131 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:43.149533 10131 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:43.149752 10131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:43.152328 10131 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed924fcf23344bfca8c4e03a2c87726e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44289 } }
I20260923 20:46:43.153012 10131 leader_election.cc:302] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed924fcf23344bfca8c4e03a2c87726e; no voters:
I20260923 20:46:43.153805 10131 leader_election.cc:288] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:43.154119 10136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:43.154990 10136 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [term 1 LEADER]: Becoming Leader. State: Replica: ed924fcf23344bfca8c4e03a2c87726e, State: Running, Role: LEADER
I20260923 20:46:43.155568 10136 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed924fcf23344bfca8c4e03a2c87726e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44289 } }
I20260923 20:46:43.156255 10131 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:43.158802 10137 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed924fcf23344bfca8c4e03a2c87726e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed924fcf23344bfca8c4e03a2c87726e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44289 } } }
I20260923 20:46:43.159430 10137 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:43.161784 10138 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed924fcf23344bfca8c4e03a2c87726e. Latest consensus state: current_term: 1 leader_uuid: "ed924fcf23344bfca8c4e03a2c87726e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed924fcf23344bfca8c4e03a2c87726e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44289 } } }
I20260923 20:46:43.162077 10142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:43.162053 10138 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:43.167588 10142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:43.178922 10142 catalog_manager.cc:1347] Generated new cluster ID: bc4b07544ab545538b4b26b4a2f3c5ac
I20260923 20:46:43.179067 10142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:43.208942 10142 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:43.209383 10142 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:43.218623 10142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e: Generated new TSK 0
I20260923 20:46:43.219470 10142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:46:43.229879 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44289
--builtin_ntp_servers=127.5.71.148:39651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 20:46:43.573544 10155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:43.573880 10155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:43.574090 10155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:43.582230 10155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:43.582393 10155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:43.582556 10155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:43.592327 10155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:43.593701 10155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:43.595547 10155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:43.609923 10160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:43.610033 10161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:43.609937 10163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:43.610786 10155 server_base.cc:1034] running on GCE node
I20260923 20:46:43.611403 10155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:43.612460 10155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:43.613694 10155 hybrid_clock.cc:648] HybridClock initialized: now 1790196403613611 us; error 79 us; skew 500 ppm
I20260923 20:46:43.614049 10155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:43.616568 10155 webserver.cc:460] Webserver started at http://127.5.71.129:46545/ using document root <none> and password file <none>
I20260923 20:46:43.617286 10155 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:43.617426 10155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:43.617748 10155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:43.620718 10155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "32111bce784b4529af78af5ea9b8b909"
format_stamp: "Formatted at 2026-09-23 20:46:43 on dist-test-slave-dn29"
server_key: "3b5cc1a2297b626ac3a7913cdf166f33"
server_key_iv: "4eb7fd0327e6402ddd8ac2f2d5e6c0a1"
server_key_version: "encryptionkey@0"
I20260923 20:46:43.621513 10155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "32111bce784b4529af78af5ea9b8b909"
format_stamp: "Formatted at 2026-09-23 20:46:43 on dist-test-slave-dn29"
server_key: "3b5cc1a2297b626ac3a7913cdf166f33"
server_key_iv: "4eb7fd0327e6402ddd8ac2f2d5e6c0a1"
server_key_version: "encryptionkey@0"
I20260923 20:46:43.626842 10155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:43.630549 10169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:43.632035 10155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:43.632172 10155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "32111bce784b4529af78af5ea9b8b909"
format_stamp: "Formatted at 2026-09-23 20:46:43 on dist-test-slave-dn29"
server_key: "3b5cc1a2297b626ac3a7913cdf166f33"
server_key_iv: "4eb7fd0327e6402ddd8ac2f2d5e6c0a1"
server_key_version: "encryptionkey@0"
I20260923 20:46:43.632373 10155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:43.638083 10155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:43.638893 10155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:43.639128 10155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:43.640228 10155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:43.641775 10155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:43.641872 10155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:43.641955 10155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:43.642019 10155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:43.676873 10155 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:43929
I20260923 20:46:43.676988 10281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:43929 every 8 connection(s)
I20260923 20:46:43.678117 10155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:43.681736 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10155
I20260923 20:46:43.681990 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:43.682830 5406 external_mini_cluster.cc:1428] Setting key 1176eb8803514840e98dbb16f53c4519
I20260923 20:46:43.685588 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44289
--builtin_ntp_servers=127.5.71.148:39651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 20:46:43.687286 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:43.680849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:47887 (local address 127.5.71.190:44289)
0923 20:46:43.681300 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:43.681305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:43.682796 (+ 1491us) server_negotiation.cc:407] Connection header received
0923 20:46:43.684807 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:43.684812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:43.684883 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:43.685005 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:43.686512 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:43.686517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:43.686520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:43.686536 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:43.686578 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:43.686590 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:43.686688 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:43.686955 (+ 267us) server_negotiation.cc:299] Negotiation successful
0923 20:46:43.687027 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":219,"threads_started":1}
I20260923 20:46:43.688577 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:43.681549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44289 (local address 127.5.71.129:47887)
0923 20:46:43.682642 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:43.682678 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:43.684518 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:43.685253 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:43.685262 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:43.685361 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:43.686109 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:43.686123 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:43.686180 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:43.686302 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:43.686691 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:43.686705 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:43.686718 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 20:46:43.687009 (+ 291us) client_negotiation.cc:240] Negotiation successful
0923 20:46:43.687294 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":386,"threads_started":1}
I20260923 20:46:43.691409 10282 heartbeater.cc:347] Connected to a master server at 127.5.71.190:44289
I20260923 20:46:43.691743 10282 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:43.692373 10282 heartbeater.cc:511] Master 127.5.71.190:44289 requested a full tablet report, sending...
I20260923 20:46:43.694664 10096 ts_manager.cc:194] Registered new tserver with Master: 32111bce784b4529af78af5ea9b8b909 (127.5.71.129:43929)
W20260923 20:46:44.018604 10286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:44.018913 10286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:44.019006 10286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:44.026341 10286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:44.026453 10286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:44.026566 10286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:44.035950 10286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:44.037321 10286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:44.039108 10286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:44.053166 10291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:44.053249 10292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:44.053670 10294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:44.054150 10286 server_base.cc:1034] running on GCE node
I20260923 20:46:44.054774 10286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:44.055758 10286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:44.057030 10286 hybrid_clock.cc:648] HybridClock initialized: now 1790196404056946 us; error 61 us; skew 500 ppm
I20260923 20:46:44.057444 10286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:44.059901 10286 webserver.cc:460] Webserver started at http://127.5.71.130:45737/ using document root <none> and password file <none>
I20260923 20:46:44.060595 10286 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:44.060689 10286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:44.060957 10286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:44.063834 10286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "87ea1bf7063e42d78ea260e4991a2eec"
format_stamp: "Formatted at 2026-09-23 20:46:44 on dist-test-slave-dn29"
server_key: "8c99cc7f1118ef177da625cdbbcd5151"
server_key_iv: "9c8d7b652f1659b7a18efe14960b5199"
server_key_version: "encryptionkey@0"
I20260923 20:46:44.064603 10286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "87ea1bf7063e42d78ea260e4991a2eec"
format_stamp: "Formatted at 2026-09-23 20:46:44 on dist-test-slave-dn29"
server_key: "8c99cc7f1118ef177da625cdbbcd5151"
server_key_iv: "9c8d7b652f1659b7a18efe14960b5199"
server_key_version: "encryptionkey@0"
I20260923 20:46:44.070046 10286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:44.074325 10300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:44.076192 10286 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:46:44.076402 10286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "87ea1bf7063e42d78ea260e4991a2eec"
format_stamp: "Formatted at 2026-09-23 20:46:44 on dist-test-slave-dn29"
server_key: "8c99cc7f1118ef177da625cdbbcd5151"
server_key_iv: "9c8d7b652f1659b7a18efe14960b5199"
server_key_version: "encryptionkey@0"
I20260923 20:46:44.076709 10286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:44.083923 10286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:44.084925 10286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:44.085248 10286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:44.086396 10286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:44.088208 10286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:44.088339 10286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:44.088485 10286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:44.088586 10286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:44.125000 10412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38903 every 8 connection(s)
I20260923 20:46:44.125001 10286 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38903
I20260923 20:46:44.126895 10286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:44.129124 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10286
I20260923 20:46:44.129299 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:44.129846 5406 external_mini_cluster.cc:1428] Setting key a6b3e6553b32c53d578c0fe791e77b7b
I20260923 20:46:44.133150 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44289
--builtin_ntp_servers=127.5.71.148:39651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 20:46:44.137422 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.130611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:46645 (local address 127.5.71.190:44289)
0923 20:46:44.130800 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.130805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.131934 (+ 1129us) server_negotiation.cc:407] Connection header received
0923 20:46:44.134584 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.134590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.134676 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.134786 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.136551 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.136558 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.136564 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.136587 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.136629 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.136645 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.136792 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.137025 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.137092 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 20:46:44.138765 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.130512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44289 (local address 127.5.71.130:46645)
0923 20:46:44.131743 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.131834 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.134256 (+ 2422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.135200 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.135213 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.135347 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.136164 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.136177 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.136236 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.136361 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.136789 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.136802 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.136813 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.137095 (+ 282us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.137383 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":368,"threads_started":1}
I20260923 20:46:44.140344 10413 heartbeater.cc:347] Connected to a master server at 127.5.71.190:44289
I20260923 20:46:44.140652 10413 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:44.141341 10413 heartbeater.cc:511] Master 127.5.71.190:44289 requested a full tablet report, sending...
I20260923 20:46:44.142829 10096 ts_manager.cc:194] Registered new tserver with Master: 87ea1bf7063e42d78ea260e4991a2eec (127.5.71.130:38903)
W20260923 20:46:44.468744 10416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:44.469112 10416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:44.469272 10416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:44.476874 10416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:44.477025 10416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:44.477208 10416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:44.486800 10416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:44289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:44.488392 10416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:44.490557 10416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:44.504499 10421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:44.504508 10424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:44.504527 10422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:44.505508 10416 server_base.cc:1034] running on GCE node
I20260923 20:46:44.506247 10416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:44.507323 10416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:44.508551 10416 hybrid_clock.cc:648] HybridClock initialized: now 1790196404508495 us; error 51 us; skew 500 ppm
I20260923 20:46:44.508922 10416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:44.511641 10416 webserver.cc:460] Webserver started at http://127.5.71.131:34051/ using document root <none> and password file <none>
I20260923 20:46:44.512691 10416 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:44.512854 10416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:44.513248 10416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:44.517398 10416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "1dc58588a06e477e81e658e5328edc2c"
format_stamp: "Formatted at 2026-09-23 20:46:44 on dist-test-slave-dn29"
server_key: "be612cd0fdfe36eb24fec439772b60f5"
server_key_iv: "e30383d716b652d2416c183e7693d196"
server_key_version: "encryptionkey@0"
I20260923 20:46:44.518440 10416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "1dc58588a06e477e81e658e5328edc2c"
format_stamp: "Formatted at 2026-09-23 20:46:44 on dist-test-slave-dn29"
server_key: "be612cd0fdfe36eb24fec439772b60f5"
server_key_iv: "e30383d716b652d2416c183e7693d196"
server_key_version: "encryptionkey@0"
I20260923 20:46:44.524525 10416 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:46:44.528530 10430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:44.529892 10416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:44.530062 10416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "1dc58588a06e477e81e658e5328edc2c"
format_stamp: "Formatted at 2026-09-23 20:46:44 on dist-test-slave-dn29"
server_key: "be612cd0fdfe36eb24fec439772b60f5"
server_key_iv: "e30383d716b652d2416c183e7693d196"
server_key_version: "encryptionkey@0"
I20260923 20:46:44.530330 10416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:44.536546 10416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:44.537367 10416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:44.537633 10416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:44.538582 10416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:44.540060 10416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:44.540148 10416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:44.540287 10416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:44.540354 10416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:44.576764 10416 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:34159
I20260923 20:46:44.576855 10542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:34159 every 8 connection(s)
I20260923 20:46:44.578158 10416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:44.586484 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10416
I20260923 20:46:44.586680 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:44.586918 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.581062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:55439 (local address 127.5.71.190:44289)
0923 20:46:44.581294 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.581299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.582811 (+ 1512us) server_negotiation.cc:407] Connection header received
0923 20:46:44.584612 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.584617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.584701 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.584813 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.586132 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.586139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.586144 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.586182 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.586214 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.586227 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.586356 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.586552 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.586622 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260923 20:46:44.587327 5406 external_mini_cluster.cc:1428] Setting key 944b06fad7d41cc10ed4ee135d014adf
I20260923 20:46:44.588030 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.581554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:44289 (local address 127.5.71.131:55439)
0923 20:46:44.582634 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.582681 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.584337 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.585046 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.585055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.585159 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.585814 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.585817 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.585857 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.585943 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.586321 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.586332 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.586340 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.586634 (+ 294us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.586898 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":335,"threads_started":1}
I20260923 20:46:44.589514 10543 heartbeater.cc:347] Connected to a master server at 127.5.71.190:44289
I20260923 20:46:44.589895 10543 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:44.590543 10543 heartbeater.cc:511] Master 127.5.71.190:44289 requested a full tablet report, sending...
I20260923 20:46:44.592025 10096 ts_manager.cc:194] Registered new tserver with Master: 1dc58588a06e477e81e658e5328edc2c (127.5.71.131:34159)
I20260923 20:46:44.603363 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:46:44.610915 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.608195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60418 (local address 127.5.71.190:44289)
0923 20:46:44.608433 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.608443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.608832 (+ 389us) server_negotiation.cc:407] Connection header received
0923 20:46:44.608977 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.608981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.609033 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.609116 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.610081 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.610085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.610088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.610103 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.610132 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.610143 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.610240 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.610561 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.610607 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:46:44.615271 10096 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:46:44.623083 10096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:46:44.642966 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.636620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38903 (local address 127.0.0.1:48608)
0923 20:46:44.637723 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.637778 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.637919 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.639182 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.639190 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.639291 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.640311 (+ 1020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.640317 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.640362 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.640464 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.641913 (+ 1449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.641929 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.641946 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.642179 (+ 233us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.642314 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":2357,"threads_started":1}
I20260923 20:46:44.644656 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.636823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.5.71.130:38903)
0923 20:46:44.638472 (+ 1649us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.638485 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.638521 (+ 36us) server_negotiation.cc:407] Connection header received
0923 20:46:44.638605 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.638617 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.638918 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.639889 (+ 971us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.641563 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.641575 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.641595 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.641624 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.641690 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.641721 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.643970 (+ 2249us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.644072 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.644214 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1471,"thread_start_us":1344,"threads_started":1}
I20260923 20:46:44.650560 10348 tablet_service.cc:1467] Processing CreateTablet for tablet 338348d4903343d0a46e9502744fd453 (DEFAULT_TABLE table=test-table [id=efff5b3b8fcc4061ac87787d4f10763a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:44.652555 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.637166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34159 (local address 127.0.0.1:44034)
0923 20:46:44.643291 (+ 6125us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.643316 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.643415 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.650792 (+ 7377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.650799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.650839 (+ 40us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.651600 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.651603 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.651620 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.651677 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.652096 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.652101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.652105 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.652211 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.652270 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5976,"threads_started":1}
I20260923 20:46:44.652694 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.637360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44034 (local address 127.5.71.131:34159)
0923 20:46:44.650381 (+ 13021us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.650387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.650413 (+ 26us) server_negotiation.cc:407] Connection header received
0923 20:46:44.650469 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.650475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.650625 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.650805 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.651916 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.651922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.651933 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.651945 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.651982 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.651995 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.652187 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.652250 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.652336 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12868,"thread_start_us":12707,"threads_started":1}
I20260923 20:46:44.653489 10348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 338348d4903343d0a46e9502744fd453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:44.654397 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.637796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39138)
0923 20:46:44.646642 (+ 8846us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.646666 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.646938 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.649416 (+ 2478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.649425 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.649462 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.652951 (+ 3489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.652954 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.652969 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.653014 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.653776 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.653780 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.653784 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.653920 (+ 136us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.653965 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8665,"thread_start_us":12426,"threads_started":1}
I20260923 20:46:44.655117 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.637405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.5.71.129:43929)
0923 20:46:44.647595 (+ 10190us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.647608 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.647641 (+ 33us) server_negotiation.cc:407] Connection header received
0923 20:46:44.647717 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.647729 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.648001 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.648186 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.653450 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.653457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.653467 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.653482 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.653529 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.653544 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.653720 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.654682 (+ 962us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.654764 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9946,"thread_start_us":17293,"threads_started":1}
I20260923 20:46:44.657935 10478 tablet_service.cc:1467] Processing CreateTablet for tablet 338348d4903343d0a46e9502744fd453 (DEFAULT_TABLE table=test-table [id=efff5b3b8fcc4061ac87787d4f10763a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:44.659044 10217 tablet_service.cc:1467] Processing CreateTablet for tablet 338348d4903343d0a46e9502744fd453 (DEFAULT_TABLE table=test-table [id=efff5b3b8fcc4061ac87787d4f10763a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:46:44.659657 10478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 338348d4903343d0a46e9502744fd453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:44.660568 10217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 338348d4903343d0a46e9502744fd453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:44.669093 10561 tablet_bootstrap.cc:492] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Bootstrap starting.
I20260923 20:46:44.672595 10561 tablet_bootstrap.cc:654] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:44.673959 10561 log.cc:824] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:44.676342 10563 tablet_bootstrap.cc:492] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Bootstrap starting.
I20260923 20:46:44.677335 10561 tablet_bootstrap.cc:492] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: No bootstrap required, opened a new log
I20260923 20:46:44.677613 10561 ts_tablet_manager.cc:1397] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260923 20:46:44.681711 10563 tablet_bootstrap.cc:654] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:44.683323 10563 log.cc:824] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:44.688743 10564 tablet_bootstrap.cc:492] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: Bootstrap starting.
I20260923 20:46:44.693678 10561 raft_consensus.cc:348] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.694074 10561 raft_consensus.cc:374] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:44.694178 10561 raft_consensus.cc:729] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ea1bf7063e42d78ea260e4991a2eec, State: Initialized, Role: FOLLOWER
I20260923 20:46:44.694852 10564 tablet_bootstrap.cc:654] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:44.694823 10561 consensus_queue.cc:260] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.696372 10564 log.cc:824] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:44.696870 10413 heartbeater.cc:503] Master 127.5.71.190:44289 was elected leader, sending a full tablet report...
I20260923 20:46:44.697701 10561 ts_tablet_manager.cc:1428] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Time spent starting tablet: real 0.020s user 0.011s sys 0.005s
I20260923 20:46:44.697973 10282 heartbeater.cc:503] Master 127.5.71.190:44289 was elected leader, sending a full tablet report...
I20260923 20:46:44.699591 10563 tablet_bootstrap.cc:492] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: No bootstrap required, opened a new log
I20260923 20:46:44.699968 10563 ts_tablet_manager.cc:1397] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.010s
I20260923 20:46:44.707152 10564 tablet_bootstrap.cc:492] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: No bootstrap required, opened a new log
I20260923 20:46:44.707475 10564 ts_tablet_manager.cc:1397] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.000s
I20260923 20:46:44.708415 10563 raft_consensus.cc:348] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.708807 10563 raft_consensus.cc:374] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:44.708912 10563 raft_consensus.cc:729] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc58588a06e477e81e658e5328edc2c, State: Initialized, Role: FOLLOWER
I20260923 20:46:44.709470 10563 consensus_queue.cc:260] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.710633 10543 heartbeater.cc:503] Master 127.5.71.190:44289 was elected leader, sending a full tablet report...
I20260923 20:46:44.711196 10563 ts_tablet_manager.cc:1428] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260923 20:46:44.717060 10564 raft_consensus.cc:348] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.717489 10564 raft_consensus.cc:374] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:44.717608 10564 raft_consensus.cc:729] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32111bce784b4529af78af5ea9b8b909, State: Initialized, Role: FOLLOWER
I20260923 20:46:44.718206 10564 consensus_queue.cc:260] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.720268 10564 ts_tablet_manager.cc:1428] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
W20260923 20:46:44.830029 10544 tablet.cc:2369] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:44.855422 10569 raft_consensus.cc:482] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:44.855973 10569 raft_consensus.cc:504] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.857986 10569 leader_election.cc:288] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87ea1bf7063e42d78ea260e4991a2eec (127.5.71.130:38903), 1dc58588a06e477e81e658e5328edc2c (127.5.71.131:34159)
I20260923 20:46:44.861708 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.858473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:37845 (local address 127.5.71.131:34159)
0923 20:46:44.858672 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.858681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.859369 (+ 688us) server_negotiation.cc:407] Connection header received
0923 20:46:44.859674 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.859692 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.859754 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.859916 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.861021 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.861028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.861033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.861052 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.861093 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.861106 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.861253 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.861336 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.861404 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:46:44.861887 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.858584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:49657 (local address 127.5.71.130:38903)
0923 20:46:44.858800 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.858807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.859938 (+ 1131us) server_negotiation.cc:407] Connection header received
0923 20:46:44.860092 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.860096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.860143 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.860247 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.861159 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.861164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.861170 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.861193 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.861224 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.861236 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.861365 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.861520 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.861585 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 20:46:44.862021 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.858595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38903 (local address 127.5.71.129:49657)
0923 20:46:44.859782 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.859812 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.859973 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.860274 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.860279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.860304 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.860947 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.860950 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.860963 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.860998 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.861369 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.861375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.861382 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.861571 (+ 189us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.861618 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"thread_start_us":653,"threads_started":1}
I20260923 20:46:44.862421 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.858298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34159 (local address 127.5.71.129:37845)
0923 20:46:44.859224 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:44.859246 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:44.859444 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:44.859945 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:44.859950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:44.859978 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:44.860763 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:46:44.860767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:44.860792 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:46:44.860849 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:44.861216 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:44.861221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:44.861228 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 20:46:44.861386 (+ 158us) client_negotiation.cc:240] Negotiation successful
0923 20:46:44.861439 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":594,"threads_started":1}
I20260923 20:46:44.863137 10368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338348d4903343d0a46e9502744fd453" candidate_uuid: "32111bce784b4529af78af5ea9b8b909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87ea1bf7063e42d78ea260e4991a2eec" is_pre_election: true
I20260923 20:46:44.863595 10498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338348d4903343d0a46e9502744fd453" candidate_uuid: "32111bce784b4529af78af5ea9b8b909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dc58588a06e477e81e658e5328edc2c" is_pre_election: true
I20260923 20:46:44.863837 10368 raft_consensus.cc:2454] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32111bce784b4529af78af5ea9b8b909 in term 0.
I20260923 20:46:44.864257 10498 raft_consensus.cc:2454] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32111bce784b4529af78af5ea9b8b909 in term 0.
I20260923 20:46:44.864674 10171 leader_election.cc:302] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32111bce784b4529af78af5ea9b8b909, 87ea1bf7063e42d78ea260e4991a2eec; no voters:
I20260923 20:46:44.865147 10569 raft_consensus.cc:2781] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:46:44.865302 10569 raft_consensus.cc:482] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:46:44.865424 10569 raft_consensus.cc:3037] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:44.867614 10569 raft_consensus.cc:504] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
I20260923 20:46:44.868245 10569 leader_election.cc:288] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [CANDIDATE]: Term 1 election: Requested vote from peers 87ea1bf7063e42d78ea260e4991a2eec (127.5.71.130:38903), 1dc58588a06e477e81e658e5328edc2c (127.5.71.131:34159)
I20260923 20:46:44.868780 10498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338348d4903343d0a46e9502744fd453" candidate_uuid: "32111bce784b4529af78af5ea9b8b909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dc58588a06e477e81e658e5328edc2c"
I20260923 20:46:44.868777 10368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338348d4903343d0a46e9502744fd453" candidate_uuid: "32111bce784b4529af78af5ea9b8b909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87ea1bf7063e42d78ea260e4991a2eec"
I20260923 20:46:44.869009 10498 raft_consensus.cc:3037] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:44.869009 10368 raft_consensus.cc:3037] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:44.872318 10368 raft_consensus.cc:2454] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32111bce784b4529af78af5ea9b8b909 in term 1.
I20260923 20:46:44.872326 10498 raft_consensus.cc:2454] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32111bce784b4529af78af5ea9b8b909 in term 1.
I20260923 20:46:44.872756 10173 leader_election.cc:302] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dc58588a06e477e81e658e5328edc2c, 32111bce784b4529af78af5ea9b8b909; no voters:
I20260923 20:46:44.873075 10569 raft_consensus.cc:2781] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:44.873530 10569 raft_consensus.cc:686] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 1 LEADER]: Becoming Leader. State: Replica: 32111bce784b4529af78af5ea9b8b909, State: Running, Role: LEADER
I20260923 20:46:44.874027 10569 consensus_queue.cc:237] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } }
W20260923 20:46:44.879016 10414 tablet.cc:2369] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:44.879280 10094 catalog_manager.cc:5495] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32111bce784b4529af78af5ea9b8b909 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "32111bce784b4529af78af5ea9b8b909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32111bce784b4529af78af5ea9b8b909" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 43929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 } health_report { overall_health: UNKNOWN } } }
I20260923 20:46:44.898397 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.896072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39142 (local address 127.5.71.129:43929)
0923 20:46:44.896236 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.896240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.896396 (+ 156us) server_negotiation.cc:407] Connection header received
0923 20:46:44.896645 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.896724 (+ 79us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.896821 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.896928 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.897773 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.897778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.897782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.897797 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.897830 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.897842 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.897946 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.898111 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.898185 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 20:46:44.907218 10368 raft_consensus.cc:1261] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 1 FOLLOWER]: Refusing update from remote peer 32111bce784b4529af78af5ea9b8b909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:44.907217 10498 raft_consensus.cc:1261] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 1 FOLLOWER]: Refusing update from remote peer 32111bce784b4529af78af5ea9b8b909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:46:44.908236 10569 consensus_queue.cc:1035] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87ea1bf7063e42d78ea260e4991a2eec" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:44.908540 10572 consensus_queue.cc:1035] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dc58588a06e477e81e658e5328edc2c" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:44.921523 10576 mvcc.cc:205] Tried to move back new op lower bound from 7332644474488852480 to 7332644474368221184. Current Snapshot: MvccSnapshot[applied={T|T < 7332644474488852480}]
I20260923 20:46:44.939276 10094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:44.939567 10094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:46:44.943243 10094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed924fcf23344bfca8c4e03a2c87726e: Sending DeleteTablet for 3 replicas of tablet 338348d4903343d0a46e9502744fd453
I20260923 20:46:44.944535 10348 tablet_service.cc:1514] Processing DeleteTablet for tablet 338348d4903343d0a46e9502744fd453 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:44 UTC) from {username='slave'} at 127.0.0.1:48608
I20260923 20:46:44.944604 10478 tablet_service.cc:1514] Processing DeleteTablet for tablet 338348d4903343d0a46e9502744fd453 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:44 UTC) from {username='slave'} at 127.0.0.1:44034
I20260923 20:46:44.945899 10586 tablet_replica.cc:330] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: stopping tablet replica
I20260923 20:46:44.946135 10587 tablet_replica.cc:330] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: stopping tablet replica
I20260923 20:46:44.946092 10217 tablet_service.cc:1514] Processing DeleteTablet for tablet 338348d4903343d0a46e9502744fd453 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:46:44 UTC) from {username='slave'} at 127.0.0.1:39138
I20260923 20:46:44.946386 10586 raft_consensus.cc:2229] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:44.946537 10587 raft_consensus.cc:2229] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:46:44.946916 10586 raft_consensus.cc:2258] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:44.947129 10587 raft_consensus.cc:2258] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:44.949725 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10155
I20260923 20:46:44.950062 10587 ts_tablet_manager.cc:1905] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:44.953716 10586 ts_tablet_manager.cc:1905] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:44.958343 10587 ts_tablet_manager.cc:1918] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:44.958573 10587 log.cc:1196] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/338348d4903343d0a46e9502744fd453
I20260923 20:46:44.959242 10587 ts_tablet_manager.cc:1939] T 338348d4903343d0a46e9502744fd453 P 1dc58588a06e477e81e658e5328edc2c: Deleting consensus metadata
I20260923 20:46:44.961371 10083 catalog_manager.cc:4841] TS 1dc58588a06e477e81e658e5328edc2c (127.5.71.131:34159): tablet 338348d4903343d0a46e9502744fd453 (table test-table [id=efff5b3b8fcc4061ac87787d4f10763a]) successfully deleted
I20260923 20:46:44.961586 10558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:44.950811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.5.71.129:43929)
0923 20:46:44.950998 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:44.951003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:44.959008 (+ 8005us) server_negotiation.cc:407] Connection header received
0923 20:46:44.959260 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:44.959267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:44.959336 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:44.959437 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:44.960718 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:44.960724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:44.960729 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:44.960749 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:44.960794 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:44.960813 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:44.960914 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:44.961186 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 20:46:44.961266 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:46:44.962780 10586 ts_tablet_manager.cc:1918] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:46:44.962890 10277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:44.962945 10586 log.cc:1196] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/338348d4903343d0a46e9502744fd453
I20260923 20:46:44.963455 10586 ts_tablet_manager.cc:1939] T 338348d4903343d0a46e9502744fd453 P 87ea1bf7063e42d78ea260e4991a2eec: Deleting consensus metadata
I20260923 20:46:44.965616 10081 catalog_manager.cc:4841] TS 87ea1bf7063e42d78ea260e4991a2eec (127.5.71.130:38903): tablet 338348d4903343d0a46e9502744fd453 (table test-table [id=efff5b3b8fcc4061ac87787d4f10763a]) successfully deleted
I20260923 20:46:45.057176 10588 tablet_replica.cc:330] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909: stopping tablet replica
I20260923 20:46:45.057686 10588 raft_consensus.cc:2229] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:46:45.058419 10588 raft_consensus.cc:2258] T 338348d4903343d0a46e9502744fd453 P 32111bce784b4529af78af5ea9b8b909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:46:45.060446 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10155
I20260923 20:46:45.079085 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10286
W20260923 20:46:45.079092 10082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:43929 (error 108)
I20260923 20:46:45.081099 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.080599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39160)
0923 20:46:45.080783 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.080937 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260923 20:46:45.081494 10082 catalog_manager.cc:4584] TS 32111bce784b4529af78af5ea9b8b909 (127.5.71.129:43929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 338348d4903343d0a46e9502744fd453: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
I20260923 20:46:45.082584 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.079993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.5.71.130:38903)
0923 20:46:45.080167 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:45.080172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:45.080277 (+ 105us) server_negotiation.cc:407] Connection header received
0923 20:46:45.080499 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:45.080503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:45.080603 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:45.080714 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:45.081840 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:45.081846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:45.081851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:45.081875 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:45.081946 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:45.081965 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:45.082098 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:45.082195 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 20:46:45.082246 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:46:45.083343 10408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:45.135555 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.134982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39176)
0923 20:46:45.135192 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.135386 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260923 20:46:45.136756 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.136410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39178)
0923 20:46:45.136564 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.136638 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 20:46:45.189000 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10286
I20260923 20:46:45.197355 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.196910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39184)
0923 20:46:45.197111 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.197197 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260923 20:46:45.198325 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.197988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39192)
0923 20:46:45.198136 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.198211 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 20:46:45.207716 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10416
I20260923 20:46:45.210474 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.208561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.5.71.131:34159)
0923 20:46:45.208751 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:45.208756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:45.208774 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:46:45.208908 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:45.208911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:45.208988 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:45.209070 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:45.209883 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:45.209888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:45.209891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:45.209904 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:45.209935 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:45.209946 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:45.210049 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:45.210110 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 20:46:45.210153 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:46:45.211094 10538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:45.278740 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.278202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39206)
0923 20:46:45.278430 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.278554 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260923 20:46:45.280027 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.279498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:43929 (local address 127.0.0.1:39218)
0923 20:46:45.279751 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:45.279898 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:43929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260923 20:46:45.300145 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10416
I20260923 20:46:45.316282 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10065
I20260923 20:46:45.317543 10126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:45.426707 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10065
2026-09-23T20:46:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2808 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 20:46:45.470763 5406 mini_oidc.cc:152] Starting JWKS server
I20260923 20:46:45.473469 5406 webserver.cc:460] Webserver started at https://127.0.0.1:33565/ using document root <none> and password file <none>
I20260923 20:46:45.473800 5406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 20:46:45.474478 5406 webserver.cc:460] Webserver started at http://127.0.0.1:40513/ using document root <none> and password file <none>
2026-09-23T20:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:45Z Disabled control of system clock
I20260923 20:46:45.502779 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43307
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:34907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:43307
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33565/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/testchainca.cert with env {}
W20260923 20:46:45.848328 10600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:45.848663 10600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:45.848773 10600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:45.857067 10600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:45.857187 10600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:45.857228 10600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:45.857290 10600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:45.857339 10600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 20:46:45.857399 10600 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 20:46:45.857435 10600 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33565/jwks.json
I20260923 20:46:45.868867 10600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:43307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33565/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:45.870193 10600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:45.871507 10600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:45.883994 10608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:45.884536 10606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:45.885067 10605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:45.885154 10600 server_base.cc:1034] running on GCE node
I20260923 20:46:45.886071 10600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:45.888099 10600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:45.889387 10600 hybrid_clock.cc:648] HybridClock initialized: now 1790196405889315 us; error 66 us; skew 500 ppm
I20260923 20:46:45.889920 10600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:45.892758 10600 webserver.cc:460] Webserver started at http://127.5.71.190:33435/ using document root <none> and password file <none>
I20260923 20:46:45.893615 10600 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:45.893790 10600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:45.894193 10600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:45.897681 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "e4fc783dd5b94d6aa49569c78a045074"
format_stamp: "Formatted at 2026-09-23 20:46:45 on dist-test-slave-dn29"
server_key: "3f1da0529a9c7a177662e818a0792571"
server_key_iv: "b79f64ee489726a037aa4ebcf0bb08d0"
server_key_version: "encryptionkey@0"
I20260923 20:46:45.898566 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "e4fc783dd5b94d6aa49569c78a045074"
format_stamp: "Formatted at 2026-09-23 20:46:45 on dist-test-slave-dn29"
server_key: "3f1da0529a9c7a177662e818a0792571"
server_key_iv: "b79f64ee489726a037aa4ebcf0bb08d0"
server_key_version: "encryptionkey@0"
I20260923 20:46:45.904842 10600 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260923 20:46:45.908758 10614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:45.910385 10600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260923 20:46:45.910600 10600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "e4fc783dd5b94d6aa49569c78a045074"
format_stamp: "Formatted at 2026-09-23 20:46:45 on dist-test-slave-dn29"
server_key: "3f1da0529a9c7a177662e818a0792571"
server_key_iv: "b79f64ee489726a037aa4ebcf0bb08d0"
server_key_version: "encryptionkey@0"
I20260923 20:46:45.910847 10600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:45.932747 10600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:45.933763 10600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:45.934091 10600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:45.953419 10600 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:43307
I20260923 20:46:45.953449 10665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:43307 every 8 connection(s)
I20260923 20:46:45.954890 10600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:45.959220 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:45.965468 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10600
I20260923 20:46:45.965734 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:45.966620 5406 external_mini_cluster.cc:1428] Setting key 15378a78b0b6503d5c48c2328a530f5b
I20260923 20:46:45.974042 10666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074: Bootstrap starting.
I20260923 20:46:45.977253 10666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:45.978333 10666 log.cc:824] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:45.981402 10666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074: No bootstrap required, opened a new log
I20260923 20:46:45.989282 10666 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fc783dd5b94d6aa49569c78a045074" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43307 } }
I20260923 20:46:45.989719 10666 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:45.989846 10666 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fc783dd5b94d6aa49569c78a045074, State: Initialized, Role: FOLLOWER
I20260923 20:46:45.990583 10666 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fc783dd5b94d6aa49569c78a045074" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43307 } }
I20260923 20:46:45.990837 10666 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:45.990978 10666 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:45.991189 10666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:45.993614 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:45.968865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.5.71.190:43307)
0923 20:46:45.969928 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:45.969941 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:45.969983 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:46:45.972152 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:45.972179 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:45.972662 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:45.973090 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:45.973732 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:45.975242 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:45.976704 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:45.977456 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:45.977701 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:45.990503 (+ 12802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:45.990513 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:45.990531 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:45.991386 (+ 855us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:45.991484 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:45.991514 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:45.991628 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:45.992118 (+ 490us) server_negotiation.cc:299] Negotiation successful
0923 20:46:45.992546 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":365,"threads_started":1}
I20260923 20:46:45.993878 10666 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fc783dd5b94d6aa49569c78a045074" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43307 } }
I20260923 20:46:45.994436 10666 leader_election.cc:302] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4fc783dd5b94d6aa49569c78a045074; no voters:
I20260923 20:46:45.995054 10666 leader_election.cc:288] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:45.995250 10671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:45.997298 10666 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:45.997953 10671 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [term 1 LEADER]: Becoming Leader. State: Replica: e4fc783dd5b94d6aa49569c78a045074, State: Running, Role: LEADER
I20260923 20:46:45.998526 10671 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fc783dd5b94d6aa49569c78a045074" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43307 } }
I20260923 20:46:45.999198 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:46:46.004650 10672 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4fc783dd5b94d6aa49569c78a045074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fc783dd5b94d6aa49569c78a045074" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43307 } } }
I20260923 20:46:46.005095 10672 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:46.006080 10680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:46.005882 10673 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4fc783dd5b94d6aa49569c78a045074. Latest consensus state: current_term: 1 leader_uuid: "e4fc783dd5b94d6aa49569c78a045074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fc783dd5b94d6aa49569c78a045074" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43307 } } }
I20260923 20:46:46.006186 10673 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:46.012781 10680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:46.023025 10680 catalog_manager.cc:1347] Generated new cluster ID: 461bb0ae93da43e7b8627a8f8af69165
I20260923 20:46:46.023157 10680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:46.039496 10680 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:46.040964 10680 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:46.055742 10680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4fc783dd5b94d6aa49569c78a045074: Generated new TSK 0
I20260923 20:46:46.056737 10680 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 20:46:46.097491 10696 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 20:46:46.097813 10696 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 20:46:46.100440 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:46.074386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.5.71.190:43307)
0923 20:46:46.074604 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:46.074611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:46.075281 (+ 670us) server_negotiation.cc:407] Connection header received
0923 20:46:46.075604 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:46.075611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:46.075715 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:46.075915 (+ 200us) server_negotiation.cc:226] Negotiated authn=JWT
0923 20:46:46.076465 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.077643 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.079239 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.079975 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.080103 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:46.080737 (+ 634us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 20:46:46.099705 (+ 18968us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 20:46:46.100133 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40702: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260923 20:46:46.100570 10669 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40702: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 20:46:46.102217 10695 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:46.074377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43307 (local address 127.0.0.1:40702)
0923 20:46:46.075159 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:46.075181 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:46.075358 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:46.075937 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:46.075941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:46.075951 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0923 20:46:46.076262 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:46.076270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.077808 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.077823 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:46.079084 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:46.079096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.080155 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.080160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:46.080322 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:46.080410 (+ 88us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 20:46:46.100097 (+ 19687us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 20:46:46.100449 (+ 352us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.71.190:43307: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":419,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.71.190:43307: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33565/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 20:46:46.104614 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10600
I20260923 20:46:46.105882 10661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:46.202540 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10600
2026-09-23T20:46:46Z chronyd exiting
I20260923 20:46:46.253341 5406 test_util.cc:175] -----------------------------------------------
I20260923 20:46:46.253485 5406 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790196359367181-5406-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (787 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 20:46:46.257529 5406 mini_oidc.cc:152] Starting JWKS server
I20260923 20:46:46.259394 5406 webserver.cc:460] Webserver started at https://127.0.0.1:40423/ using document root <none> and password file <none>
I20260923 20:46:46.259635 5406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 20:46:46.260375 5406 webserver.cc:460] Webserver started at http://127.0.0.1:34803/ using document root <none> and password file <none>
2026-09-23T20:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:46Z Disabled control of system clock
I20260923 20:46:46.288756 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38279
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:38279
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40423/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/testchainca.cert with env {}
W20260923 20:46:46.622215 10703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:46.622591 10703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:46.622797 10703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:46.630148 10703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:46.630323 10703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:46.630419 10703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:46.630522 10703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:46.630611 10703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 20:46:46.630709 10703 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 20:46:46.630788 10703 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40423/jwks.json
I20260923 20:46:46.641597 10703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:38279
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40423/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:46.643007 10703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:46.644414 10703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:46.656929 10708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:46.657006 10709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:46.657119 10711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:46.657642 10703 server_base.cc:1034] running on GCE node
I20260923 20:46:46.658447 10703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:46.659967 10703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:46.661171 10703 hybrid_clock.cc:648] HybridClock initialized: now 1790196406661131 us; error 29 us; skew 500 ppm
I20260923 20:46:46.661504 10703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:46.665030 10703 webserver.cc:460] Webserver started at http://127.5.71.190:42279/ using document root <none> and password file <none>
I20260923 20:46:46.665753 10703 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:46.665900 10703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:46.666246 10703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:46.669229 10703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "4ab1b64fee694faa9a27ebff02a38620"
format_stamp: "Formatted at 2026-09-23 20:46:46 on dist-test-slave-dn29"
server_key: "0af60a65aa5eba6c5f37da0df3da99bc"
server_key_iv: "90e172aed23850471eb602ea039cd224"
server_key_version: "encryptionkey@0"
I20260923 20:46:46.670044 10703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "4ab1b64fee694faa9a27ebff02a38620"
format_stamp: "Formatted at 2026-09-23 20:46:46 on dist-test-slave-dn29"
server_key: "0af60a65aa5eba6c5f37da0df3da99bc"
server_key_iv: "90e172aed23850471eb602ea039cd224"
server_key_version: "encryptionkey@0"
I20260923 20:46:46.675446 10703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260923 20:46:46.679394 10717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:46.681219 10703 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:46:46.681449 10703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "4ab1b64fee694faa9a27ebff02a38620"
format_stamp: "Formatted at 2026-09-23 20:46:46 on dist-test-slave-dn29"
server_key: "0af60a65aa5eba6c5f37da0df3da99bc"
server_key_iv: "90e172aed23850471eb602ea039cd224"
server_key_version: "encryptionkey@0"
I20260923 20:46:46.681704 10703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:46.713100 10703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:46.714061 10703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:46.714416 10703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:46.734632 10703 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:38279
I20260923 20:46:46.734656 10768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:38279 every 8 connection(s)
I20260923 20:46:46.736120 10703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:46.739391 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10703
I20260923 20:46:46.739642 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:46.740509 10769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:46.740828 5406 external_mini_cluster.cc:1428] Setting key 20dc204f80749046751df027d9f0b396
I20260923 20:46:46.755891 10769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620: Bootstrap starting.
I20260923 20:46:46.758284 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:46.743027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.5.71.190:38279)
0923 20:46:46.744274 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:46.744286 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:46.744334 (+ 48us) server_negotiation.cc:407] Connection header received
0923 20:46:46.747202 (+ 2868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:46.747232 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:46.747884 (+ 652us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:46.748296 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:46.749085 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.750708 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.752860 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.753700 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.753873 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:46.755322 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:46.755330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:46.755347 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:46.756172 (+ 825us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:46.756271 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:46.756294 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:46.756394 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:46.756746 (+ 352us) server_negotiation.cc:299] Negotiation successful
0923 20:46:46.757174 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":460,"threads_started":1}
I20260923 20:46:46.759572 10769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:46.761240 10769 log.cc:824] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:46.763301 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:46:46.764827 10769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620: No bootstrap required, opened a new log
I20260923 20:46:46.774046 10769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab1b64fee694faa9a27ebff02a38620" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38279 } }
I20260923 20:46:46.774518 10769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:46.774641 10769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab1b64fee694faa9a27ebff02a38620, State: Initialized, Role: FOLLOWER
I20260923 20:46:46.775429 10769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab1b64fee694faa9a27ebff02a38620" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38279 } }
I20260923 20:46:46.775774 10769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:46.775955 10769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:46.776114 10769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:46.778520 10769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab1b64fee694faa9a27ebff02a38620" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38279 } }
I20260923 20:46:46.779037 10769 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab1b64fee694faa9a27ebff02a38620; no voters:
I20260923 20:46:46.779645 10769 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:46.779934 10774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:46.780855 10774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [term 1 LEADER]: Becoming Leader. State: Replica: 4ab1b64fee694faa9a27ebff02a38620, State: Running, Role: LEADER
I20260923 20:46:46.781473 10774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab1b64fee694faa9a27ebff02a38620" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38279 } }
I20260923 20:46:46.782615 10769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:46.784718 10776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab1b64fee694faa9a27ebff02a38620. Latest consensus state: current_term: 1 leader_uuid: "4ab1b64fee694faa9a27ebff02a38620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab1b64fee694faa9a27ebff02a38620" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38279 } } }
I20260923 20:46:46.785300 10776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:46.790060 10775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ab1b64fee694faa9a27ebff02a38620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab1b64fee694faa9a27ebff02a38620" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38279 } } }
I20260923 20:46:46.790467 10775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:46.792297 10784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:46.797655 10784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:46.807240 10784 catalog_manager.cc:1347] Generated new cluster ID: 9dffdf75deb447b78d167ebb707dd8ac
I20260923 20:46:46.807365 10784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:46.833142 10784 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:46.834638 10784 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:46.843820 10784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ab1b64fee694faa9a27ebff02a38620: Generated new TSK 0
I20260923 20:46:46.844821 10784 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 20:46:46.883535 10799 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 20:46:46.883682 10799 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 20:46:46.885037 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:46.862727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52974 (local address 127.5.71.190:38279)
0923 20:46:46.862912 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:46.862919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:46.863166 (+ 247us) server_negotiation.cc:407] Connection header received
0923 20:46:46.863503 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:46.863509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:46.863616 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:46.863814 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0923 20:46:46.864397 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.865558 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.867082 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.867817 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.867949 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:46.868486 (+ 537us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 20:46:46.884223 (+ 15737us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 20:46:46.884664 (+ 441us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52974: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260923 20:46:46.885170 10772 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52974: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 20:46:46.885324 10798 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:46.862333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38279 (local address 127.0.0.1:52974)
0923 20:46:46.863031 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:46.863081 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:46.863283 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:46.863896 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:46.863901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:46.863913 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0923 20:46:46.864232 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:46.864243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.865838 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.865842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:46.866929 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:46.866941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:46.867984 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:46.867989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:46.868170 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:46.868208 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 20:46:46.884526 (+ 16318us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 20:46:46.884851 (+ 325us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.71.190:38279: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":587,"threads_started":1}
I20260923 20:46:46.886691 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10703
I20260923 20:46:46.888006 10764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:46.984519 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10703
2026-09-23T20:46:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (783 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 20:46:47.040719 5406 mini_oidc.cc:152] Starting JWKS server
I20260923 20:46:47.042677 5406 webserver.cc:460] Webserver started at https://127.0.0.1:42119/ using document root <none> and password file <none>
I20260923 20:46:47.042951 5406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 20:46:47.043598 5406 webserver.cc:460] Webserver started at http://127.0.0.1:38951/ using document root <none> and password file <none>
2026-09-23T20:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:47Z Disabled control of system clock
I20260923 20:46:47.072737 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42833
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:36359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:42833
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42119/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 20:46:47.405695 10806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:47.406014 10806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:47.406101 10806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:47.413477 10806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:47.413590 10806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:47.413640 10806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:47.413691 10806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:47.413728 10806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 20:46:47.413807 10806 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 20:46:47.413851 10806 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42119/jwks.json
I20260923 20:46:47.423231 10806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:42833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42119/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:47.424573 10806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:47.426349 10806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:47.438628 10812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:47.438664 10811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:47.438778 10814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:47.439477 10806 server_base.cc:1034] running on GCE node
I20260923 20:46:47.440539 10806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:47.443531 10806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:47.445039 10806 hybrid_clock.cc:648] HybridClock initialized: now 1790196407444961 us; error 103 us; skew 500 ppm
I20260923 20:46:47.445449 10806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:47.448148 10806 webserver.cc:460] Webserver started at http://127.5.71.190:39017/ using document root <none> and password file <none>
I20260923 20:46:47.448920 10806 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:47.449055 10806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:47.449407 10806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:47.452600 10806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "9195f6c3b42f474c9cf32f98c668ef73"
format_stamp: "Formatted at 2026-09-23 20:46:47 on dist-test-slave-dn29"
server_key: "6d770e540510b54c94d79728bce90c87"
server_key_iv: "67a7a142ecdbf0f81bfca063dd63f9b1"
server_key_version: "encryptionkey@0"
I20260923 20:46:47.453445 10806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "9195f6c3b42f474c9cf32f98c668ef73"
format_stamp: "Formatted at 2026-09-23 20:46:47 on dist-test-slave-dn29"
server_key: "6d770e540510b54c94d79728bce90c87"
server_key_iv: "67a7a142ecdbf0f81bfca063dd63f9b1"
server_key_version: "encryptionkey@0"
I20260923 20:46:47.459136 10806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:47.462915 10820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:47.464607 10806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:47.464818 10806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "9195f6c3b42f474c9cf32f98c668ef73"
format_stamp: "Formatted at 2026-09-23 20:46:47 on dist-test-slave-dn29"
server_key: "6d770e540510b54c94d79728bce90c87"
server_key_iv: "67a7a142ecdbf0f81bfca063dd63f9b1"
server_key_version: "encryptionkey@0"
I20260923 20:46:47.465058 10806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:47.485906 10806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:47.486856 10806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:47.487188 10806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:47.507483 10806 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:42833
I20260923 20:46:47.507503 10871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:42833 every 8 connection(s)
I20260923 20:46:47.508996 10806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:47.513445 10872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:47.514225 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10806
I20260923 20:46:47.514554 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:47.515312 5406 external_mini_cluster.cc:1428] Setting key 475d247e2f3a9f66befdbd0296c326ad
I20260923 20:46:47.527087 10872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73: Bootstrap starting.
I20260923 20:46:47.529418 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:47.517379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.5.71.190:42833)
0923 20:46:47.518656 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:47.518672 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:47.518803 (+ 131us) server_negotiation.cc:407] Connection header received
0923 20:46:47.520575 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:47.520599 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:47.521127 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:47.521535 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:47.522229 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:47.523482 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:47.524933 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:47.525650 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:47.525858 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:47.527036 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:47.527043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:47.527058 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:47.527670 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:47.527756 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:47.527777 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:47.527960 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:47.528181 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 20:46:47.528483 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":561,"threads_started":1}
I20260923 20:46:47.530753 10872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:47.531941 10872 log.cc:824] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:47.533696 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:46:47.535003 10872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73: No bootstrap required, opened a new log
I20260923 20:46:47.543246 10872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9195f6c3b42f474c9cf32f98c668ef73" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42833 } }
I20260923 20:46:47.544044 10872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:47.544189 10872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9195f6c3b42f474c9cf32f98c668ef73, State: Initialized, Role: FOLLOWER
I20260923 20:46:47.544889 10872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9195f6c3b42f474c9cf32f98c668ef73" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42833 } }
I20260923 20:46:47.545120 10872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:47.545219 10872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:47.545354 10872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:47.547525 10872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9195f6c3b42f474c9cf32f98c668ef73" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42833 } }
I20260923 20:46:47.548128 10872 leader_election.cc:302] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9195f6c3b42f474c9cf32f98c668ef73; no voters:
I20260923 20:46:47.548905 10872 leader_election.cc:288] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:47.549113 10877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:47.549918 10877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [term 1 LEADER]: Becoming Leader. State: Replica: 9195f6c3b42f474c9cf32f98c668ef73, State: Running, Role: LEADER
I20260923 20:46:47.550487 10877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9195f6c3b42f474c9cf32f98c668ef73" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42833 } }
I20260923 20:46:47.551244 10872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:47.553473 10878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9195f6c3b42f474c9cf32f98c668ef73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9195f6c3b42f474c9cf32f98c668ef73" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42833 } } }
I20260923 20:46:47.553521 10879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9195f6c3b42f474c9cf32f98c668ef73. Latest consensus state: current_term: 1 leader_uuid: "9195f6c3b42f474c9cf32f98c668ef73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9195f6c3b42f474c9cf32f98c668ef73" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42833 } } }
I20260923 20:46:47.553995 10878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:47.554045 10879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:47.556672 10883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:47.562620 10883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:47.572256 10883 catalog_manager.cc:1347] Generated new cluster ID: 19231495c8c947209853610275efffa6
I20260923 20:46:47.572383 10883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:47.597432 10883 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:47.598809 10883 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:47.608224 10883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9195f6c3b42f474c9cf32f98c668ef73: Generated new TSK 0
I20260923 20:46:47.609136 10883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:46:47.637065 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:47.627283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.5.71.190:42833)
0923 20:46:47.627491 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:47.627498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:47.628076 (+ 578us) server_negotiation.cc:407] Connection header received
0923 20:46:47.628351 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:47.628358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:47.628452 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:47.628626 (+ 174us) server_negotiation.cc:226] Negotiated authn=JWT
0923 20:46:47.629148 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:47.630334 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:47.632139 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:47.632804 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:47.632978 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:47.633600 (+ 622us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 20:46:47.636310 (+ 2710us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 20:46:47.636716 (+ 406us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60078: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":69}
W20260923 20:46:47.637221 10875 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60078: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 20:46:47.638989 10901 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:47.627243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42833 (local address 127.0.0.1:60078)
0923 20:46:47.627973 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:47.627991 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:47.628131 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:47.628679 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:47.628685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:47.628700 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0923 20:46:47.628987 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:47.628996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:47.630524 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:47.630529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:47.631951 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:47.631965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:47.633048 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:47.633059 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:47.633226 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:47.633266 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 20:46:47.636598 (+ 3332us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 20:46:47.638553 (+ 1955us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.71.190:42833: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42119/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":1625,"thread_start_us":415,"threads_started":1}
I20260923 20:46:47.640198 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10806
I20260923 20:46:47.641901 10867 generic_service.cc:196] Checking for leaks (request via RPC)
E20260923 20:46:47.649654 10902 webserver.cc:533] Webserver: sslize failed:
E20260923 20:46:47.649789 10902 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 20:46:47.738232 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10806
2026-09-23T20:46:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (754 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T20:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:47Z Disabled control of system clock
I20260923 20:46:47.823433 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45695
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:45439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45695
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/testchainca.cert with env {}
W20260923 20:46:48.171013 10907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:48.171365 10907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:48.171517 10907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:48.178997 10907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:48.179117 10907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:48.179191 10907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:48.179286 10907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:48.179378 10907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 20:46:48.179489 10907 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 20:46:48.179551 10907 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 20:46:48.191320 10907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:48.192857 10907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:48.194319 10907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:48.208199 10912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:48.208155 10915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:48.208238 10913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:48.208915 10907 server_base.cc:1034] running on GCE node
I20260923 20:46:48.209759 10907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:48.211295 10907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:48.212595 10907 hybrid_clock.cc:648] HybridClock initialized: now 1790196408212565 us; error 53 us; skew 500 ppm
I20260923 20:46:48.213097 10907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:46:48.215847 10907 webserver.cc:460] Webserver started at http://127.5.71.190:38723/ using document root <none> and password file <none>
I20260923 20:46:48.216596 10907 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:48.216741 10907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:48.217065 10907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:48.220116 10907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "00ab7d739a13438d80a49e15b6e505d8"
format_stamp: "Formatted at 2026-09-23 20:46:48 on dist-test-slave-dn29"
server_key: "08bc3eaba57cf93eb1c9ba8ac74ab012"
server_key_iv: "b9b9d7b1afd071eccafca6bab71b93ae"
server_key_version: "encryptionkey@0"
I20260923 20:46:48.220986 10907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "00ab7d739a13438d80a49e15b6e505d8"
format_stamp: "Formatted at 2026-09-23 20:46:48 on dist-test-slave-dn29"
server_key: "08bc3eaba57cf93eb1c9ba8ac74ab012"
server_key_iv: "b9b9d7b1afd071eccafca6bab71b93ae"
server_key_version: "encryptionkey@0"
I20260923 20:46:48.226509 10907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:46:48.230587 10921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:48.232303 10907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260923 20:46:48.232509 10907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "00ab7d739a13438d80a49e15b6e505d8"
format_stamp: "Formatted at 2026-09-23 20:46:48 on dist-test-slave-dn29"
server_key: "08bc3eaba57cf93eb1c9ba8ac74ab012"
server_key_iv: "b9b9d7b1afd071eccafca6bab71b93ae"
server_key_version: "encryptionkey@0"
I20260923 20:46:48.232752 10907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:48.255393 10907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:48.256373 10907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:48.256700 10907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:48.276261 10907 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45695
I20260923 20:46:48.276288 10972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45695 every 8 connection(s)
I20260923 20:46:48.277669 10907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:48.281720 10973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:48.286162 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 10907
I20260923 20:46:48.286396 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:48.287273 5406 external_mini_cluster.cc:1428] Setting key 229614818f56d3149be390a0ed609a38
I20260923 20:46:48.294972 10973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8: Bootstrap starting.
I20260923 20:46:48.298559 10973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:48.299717 10973 log.cc:824] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:48.302637 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:48.289756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.5.71.190:45695)
0923 20:46:48.290871 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:48.290884 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:48.290929 (+ 45us) server_negotiation.cc:407] Connection header received
0923 20:46:48.293306 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:48.293340 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:48.293871 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:48.294291 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:48.294904 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:48.296466 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:48.297830 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:48.298525 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:48.298749 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:48.299955 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:48.299964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:48.299979 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:46:48.300757 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:48.300863 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:46:48.300893 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:48.301036 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:48.301285 (+ 249us) server_negotiation.cc:299] Negotiation successful
0923 20:46:48.301657 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":392,"threads_started":1}
I20260923 20:46:48.303169 10973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8: No bootstrap required, opened a new log
I20260923 20:46:48.306524 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:46:48.311517 10973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab7d739a13438d80a49e15b6e505d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45695 } }
I20260923 20:46:48.312033 10973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:48.312144 10973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ab7d739a13438d80a49e15b6e505d8, State: Initialized, Role: FOLLOWER
I20260923 20:46:48.312793 10973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab7d739a13438d80a49e15b6e505d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45695 } }
I20260923 20:46:48.313056 10973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:48.313205 10973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:48.313405 10973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:48.316061 10973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab7d739a13438d80a49e15b6e505d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45695 } }
I20260923 20:46:48.316579 10973 leader_election.cc:302] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ab7d739a13438d80a49e15b6e505d8; no voters:
I20260923 20:46:48.317209 10973 leader_election.cc:288] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:48.317384 10978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:48.318233 10978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [term 1 LEADER]: Becoming Leader. State: Replica: 00ab7d739a13438d80a49e15b6e505d8, State: Running, Role: LEADER
I20260923 20:46:48.318909 10978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab7d739a13438d80a49e15b6e505d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45695 } }
I20260923 20:46:48.319716 10973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:48.322072 10980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ab7d739a13438d80a49e15b6e505d8. Latest consensus state: current_term: 1 leader_uuid: "00ab7d739a13438d80a49e15b6e505d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab7d739a13438d80a49e15b6e505d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45695 } } }
I20260923 20:46:48.322547 10980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:48.324610 10979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ab7d739a13438d80a49e15b6e505d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ab7d739a13438d80a49e15b6e505d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45695 } } }
I20260923 20:46:48.324985 10979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:48.325344 10984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:48.330844 10984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:48.341217 10984 catalog_manager.cc:1347] Generated new cluster ID: f17a5f4de4df418a82e82222797b4f6f
I20260923 20:46:48.341398 10984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:48.362274 10984 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:48.364341 10984 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:48.372494 10984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ab7d739a13438d80a49e15b6e505d8: Generated new TSK 0
I20260923 20:46:48.373440 10984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:46:48.511511 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:48.395449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.5.71.190:45695)
0923 20:46:48.395709 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:48.395715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:48.396231 (+ 516us) server_negotiation.cc:407] Connection header received
0923 20:46:48.396476 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:48.396480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:48.396578 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:48.396727 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0923 20:46:48.397414 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:48.398286 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:48.400415 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:48.401069 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:48.401173 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:48.402001 (+ 828us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 20:46:48.510643 (+108642us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 20:46:48.511154 (+ 511us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33960: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":95}
W20260923 20:46:48.511758 10976 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33960: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 20:46:48.512089 11002 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:48.395270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45695 (local address 127.0.0.1:33960)
0923 20:46:48.396031 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:48.396054 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:48.396227 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:48.396781 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:48.396787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:48.396802 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0923 20:46:48.397181 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:48.397194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:48.398456 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:48.398462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:48.400246 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:48.400262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:48.401269 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:48.401276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:48.401477 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:48.401528 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 20:46:48.511039 (+109511us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 20:46:48.511387 (+ 348us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.71.190:45695: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":331,"threads_started":1}
I20260923 20:46:48.516950 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 10907
I20260923 20:46:48.521724 10968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:48.633441 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 10907
2026-09-23T20:46:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (871 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (430 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (422 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:49 dist-test-slave-dn29 krb5kdc[11017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:49 dist-test-slave-dn29 krb5kdc[11017](info): set up 2 sockets
Sep 23 20:46:49 dist-test-slave-dn29 krb5kdc[11017](info): commencing operation
krb5kdc: starting...
W20260923 20:46:50.633145 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:50 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:46:50.861017 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:41997
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:41997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:51.201180 11030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:51.201495 11030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:51.201586 11030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:51.209118 11030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:51.209242 11030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:51.209290 11030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:51.209339 11030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:51.209383 11030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:51.220191 11030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:41997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:41997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:51.221531 11030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:51.222818 11030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:51.235416 11036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:51.235553 11035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:51.235699 11038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:51.236723 11030 server_base.cc:1034] running on GCE node
I20260923 20:46:51.237586 11030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:51.239140 11030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:51.240399 11030 hybrid_clock.cc:648] HybridClock initialized: now 1790196411240329 us; error 49 us; skew 500 ppm
I20260923 20:46:51.240841 11030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:51 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196411, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:51.250037 11030 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:46:51.251776 11030 webserver.cc:460] Webserver started at http://127.5.71.190:45855/ using document root <none> and password file <none>
I20260923 20:46:51.252642 11030 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:51.252763 11030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:51.253082 11030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:51.256734 11030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "9574baa58e77492da3ead90c94e098f1"
format_stamp: "Formatted at 2026-09-23 20:46:51 on dist-test-slave-dn29"
server_key: "b7c94e9c3e39c50a30e529106ba212af"
server_key_iv: "d0f9f293b47ee03a82c1d3b2f8b86c45"
server_key_version: "encryptionkey@0"
I20260923 20:46:51.257501 11030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "9574baa58e77492da3ead90c94e098f1"
format_stamp: "Formatted at 2026-09-23 20:46:51 on dist-test-slave-dn29"
server_key: "b7c94e9c3e39c50a30e529106ba212af"
server_key_iv: "d0f9f293b47ee03a82c1d3b2f8b86c45"
server_key_version: "encryptionkey@0"
I20260923 20:46:51.262938 11030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:51.266781 11045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:51.268410 11030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260923 20:46:51.268589 11030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "9574baa58e77492da3ead90c94e098f1"
format_stamp: "Formatted at 2026-09-23 20:46:51 on dist-test-slave-dn29"
server_key: "b7c94e9c3e39c50a30e529106ba212af"
server_key_iv: "d0f9f293b47ee03a82c1d3b2f8b86c45"
server_key_version: "encryptionkey@0"
I20260923 20:46:51.268776 11030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:51.285600 11030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:51.290179 11030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:51.290515 11030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:51.311547 11030 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:41997
I20260923 20:46:51.311600 11096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:41997 every 8 connection(s)
I20260923 20:46:51.313037 11030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:51.314738 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11030
I20260923 20:46:51.314953 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:51.315589 5406 external_mini_cluster.cc:1428] Setting key 9de364b61413ef201acf033a41883885
I20260923 20:46:51.318209 11097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:46:51 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:51.332119 11097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1: Bootstrap starting.
I20260923 20:46:51.336526 11097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:51.337946 11097 log.cc:824] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:51.340184 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:51.317481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.5.71.190:41997)
0923 20:46:51.318258 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:51.318271 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:51.318334 (+ 63us) server_negotiation.cc:407] Connection header received
0923 20:46:51.320055 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:51.320077 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:51.320490 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:51.320824 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:51.322575 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:51.324009 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:51.325770 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:51.326452 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:51.326586 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:51.331352 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:51.331364 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:51.331387 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:51.331444 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:51.335526 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:51.336275 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:51.336286 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:51.336298 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:51.336523 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:51.336955 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:51.336962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:51.336971 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:51.337684 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:51.338000 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:51.338443 (+ 443us) server_negotiation.cc:299] Negotiation successful
0923 20:46:51.338932 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":361,"threads_started":1}
I20260923 20:46:51.341496 11097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1: No bootstrap required, opened a new log
I20260923 20:46:51.349205 11097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9574baa58e77492da3ead90c94e098f1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41997 } }
I20260923 20:46:51.349586 11097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:51.349712 11097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9574baa58e77492da3ead90c94e098f1, State: Initialized, Role: FOLLOWER
I20260923 20:46:51.350289 11097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9574baa58e77492da3ead90c94e098f1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41997 } }
I20260923 20:46:51.350514 11097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:51.350672 11097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:51.350832 11097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:51.353411 11097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9574baa58e77492da3ead90c94e098f1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41997 } }
I20260923 20:46:51.353924 11097 leader_election.cc:302] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9574baa58e77492da3ead90c94e098f1; no voters:
I20260923 20:46:51.354547 11097 leader_election.cc:288] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:51.354715 11102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:51.355443 11102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [term 1 LEADER]: Becoming Leader. State: Replica: 9574baa58e77492da3ead90c94e098f1, State: Running, Role: LEADER
I20260923 20:46:51.356070 11102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9574baa58e77492da3ead90c94e098f1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41997 } }
I20260923 20:46:51.356906 11097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:51.359263 11103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9574baa58e77492da3ead90c94e098f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9574baa58e77492da3ead90c94e098f1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41997 } } }
I20260923 20:46:51.359439 11104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9574baa58e77492da3ead90c94e098f1. Latest consensus state: current_term: 1 leader_uuid: "9574baa58e77492da3ead90c94e098f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9574baa58e77492da3ead90c94e098f1" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41997 } } }
I20260923 20:46:51.359735 11103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:51.359730 11104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1 [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:51.361709 11110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:51.367261 11110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:51.376236 11110 catalog_manager.cc:1347] Generated new cluster ID: 7835e17f61a144bb858c5941e8f44c9f
I20260923 20:46:51.376381 11110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:51.383023 11110 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:51.384670 11110 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:51.411749 11110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9574baa58e77492da3ead90c94e098f1: Generated new TSK 0
I20260923 20:46:51.412714 11110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:46:51.480469 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41997
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:51.837630 11123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:51.837932 11123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:51.838024 11123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:51.845443 11123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:51.845558 11123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:51.845670 11123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:51.855588 11123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:51.857003 11123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:51.858606 11123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:51.872588 11129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:51.872622 11128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:51.872815 11131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:51.873340 11123 server_base.cc:1034] running on GCE node
I20260923 20:46:51.874034 11123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:51.875118 11123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:51.876354 11123 hybrid_clock.cc:648] HybridClock initialized: now 1790196411876304 us; error 51 us; skew 500 ppm
I20260923 20:46:51.876735 11123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:51 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196411, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:51.880343 11123 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:46:51.882005 11123 webserver.cc:460] Webserver started at http://127.5.71.129:37095/ using document root <none> and password file <none>
I20260923 20:46:51.882706 11123 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:51.882825 11123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:51.883103 11123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:51.886439 11123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "1a8646c967494addb2f3d03a6dacc61e"
format_stamp: "Formatted at 2026-09-23 20:46:51 on dist-test-slave-dn29"
server_key: "1d3e21302677259af44113568530423e"
server_key_iv: "c17037896e50a2b3897c8060cd6acded"
server_key_version: "encryptionkey@0"
I20260923 20:46:51.887235 11123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "1a8646c967494addb2f3d03a6dacc61e"
format_stamp: "Formatted at 2026-09-23 20:46:51 on dist-test-slave-dn29"
server_key: "1d3e21302677259af44113568530423e"
server_key_iv: "c17037896e50a2b3897c8060cd6acded"
server_key_version: "encryptionkey@0"
I20260923 20:46:51.893105 11123 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260923 20:46:51.897183 11138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:51.898710 11123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:46:51.898903 11123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "1a8646c967494addb2f3d03a6dacc61e"
format_stamp: "Formatted at 2026-09-23 20:46:51 on dist-test-slave-dn29"
server_key: "1d3e21302677259af44113568530423e"
server_key_iv: "c17037896e50a2b3897c8060cd6acded"
server_key_version: "encryptionkey@0"
I20260923 20:46:51.899139 11123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:51.918329 11123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:51.923444 11123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:51.923856 11123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:51.925004 11123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:51.926592 11123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:51.926671 11123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:51.926838 11123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:51.926903 11123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:51.965557 11123 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:34121
I20260923 20:46:51.965600 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:34121 every 8 connection(s)
I20260923 20:46:51.967025 11123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:51.972580 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11123
I20260923 20:46:51.972798 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:51.973590 5406 external_mini_cluster.cc:1428] Setting key 37140b1a0c5d0fb0de6b397caf1a6814
Sep 23 20:46:51 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196411, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:51.992430 11253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:51.969817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:51903 (local address 127.5.71.190:41997)
0923 20:46:51.970317 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:51.970322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:51.971314 (+ 992us) server_negotiation.cc:407] Connection header received
0923 20:46:51.973976 (+ 2662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:51.973981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:51.974055 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:51.974164 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:51.976025 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:51.976831 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:51.979734 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:51.980272 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:51.980388 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:51.984998 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:51.985007 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:51.985015 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:51.985064 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:51.988619 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:51.989540 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:51.989549 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:51.989556 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:51.989707 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:51.990163 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:51.990171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:51.990177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:51.990551 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:51.990705 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:51.991617 (+ 912us) server_negotiation.cc:299] Negotiation successful
0923 20:46:51.991917 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260923 20:46:51.993315 11254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:51.970256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41997 (local address 127.5.71.129:51903)
0923 20:46:51.971084 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:51.971123 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:51.973613 (+ 2490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:51.974439 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:51.974448 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:51.975097 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:51.975837 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:51.975865 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:51.977148 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:51.977158 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:51.979531 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:51.979549 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:51.980524 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:51.980533 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:51.980900 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:51.981680 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:51.981724 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:51.984723 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:51.988817 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:51.988829 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:51.988846 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:51.989376 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:51.989832 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:51.989836 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:51.989838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:51.990027 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:51.990786 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:51.990794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:51.991173 (+ 379us) client_negotiation.cc:769] Sending connection context
0923 20:46:51.991458 (+ 285us) client_negotiation.cc:240] Negotiation successful
0923 20:46:51.992017 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":370,"threads_started":1}
I20260923 20:46:51.995182 11251 heartbeater.cc:347] Connected to a master server at 127.5.71.190:41997
I20260923 20:46:51.995536 11251 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:51.996438 11251 heartbeater.cc:511] Master 127.5.71.190:41997 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:51.998847 11062 ts_manager.cc:194] Registered new tserver with Master: 1a8646c967494addb2f3d03a6dacc61e (127.5.71.129:34121)
I20260923 20:46:52.000463 11062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:51903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:46:52.028506 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41997
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:52.376348 11257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:52.376662 11257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:52.376758 11257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:52.384315 11257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:52.384429 11257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:52.384543 11257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:52.394515 11257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:52.396010 11257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:52.397462 11257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:52.411964 11262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:52.411929 11263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:52.412144 11265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:52.413194 11257 server_base.cc:1034] running on GCE node
I20260923 20:46:52.413992 11257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:52.415087 11257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:52.416314 11257 hybrid_clock.cc:648] HybridClock initialized: now 1790196412416258 us; error 55 us; skew 500 ppm
I20260923 20:46:52.416688 11257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:52 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:52.420377 11257 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:46:52.422043 11257 webserver.cc:460] Webserver started at http://127.5.71.130:43463/ using document root <none> and password file <none>
I20260923 20:46:52.422763 11257 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:52.422865 11257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:52.423211 11257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:52.426607 11257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "87616014d92144c1a58befa648bb5584"
format_stamp: "Formatted at 2026-09-23 20:46:52 on dist-test-slave-dn29"
server_key: "bf42c485ff873e3bfddc47258aaf5a92"
server_key_iv: "f5d6f76e3213e77e7447c56802cb62cb"
server_key_version: "encryptionkey@0"
I20260923 20:46:52.427466 11257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "87616014d92144c1a58befa648bb5584"
format_stamp: "Formatted at 2026-09-23 20:46:52 on dist-test-slave-dn29"
server_key: "bf42c485ff873e3bfddc47258aaf5a92"
server_key_iv: "f5d6f76e3213e77e7447c56802cb62cb"
server_key_version: "encryptionkey@0"
I20260923 20:46:52.433277 11257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260923 20:46:52.437150 11272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:52.438860 11257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:52.439096 11257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "87616014d92144c1a58befa648bb5584"
format_stamp: "Formatted at 2026-09-23 20:46:52 on dist-test-slave-dn29"
server_key: "bf42c485ff873e3bfddc47258aaf5a92"
server_key_iv: "f5d6f76e3213e77e7447c56802cb62cb"
server_key_version: "encryptionkey@0"
I20260923 20:46:52.439308 11257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:52.458202 11257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:52.464059 11257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:52.464419 11257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:52.465601 11257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:52.467514 11257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:52.467597 11257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:52.467770 11257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:52.467854 11257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260923 20:46:52.507170 11257 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:42977
I20260923 20:46:52.507213 11384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:42977 every 8 connection(s)
I20260923 20:46:52.508625 11257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:52.511601 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11257
I20260923 20:46:52.511843 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:52.512485 5406 external_mini_cluster.cc:1428] Setting key 9568eeafd5ad1411d7f66d0fa08570b8
Sep 23 20:46:52 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:52.537393 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:52.512059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:42021 (local address 127.5.71.190:41997)
0923 20:46:52.512783 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:52.512791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:52.514836 (+ 2045us) server_negotiation.cc:407] Connection header received
0923 20:46:52.517622 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:52.517628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:52.517720 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:52.517868 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:52.520172 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:52.521228 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:52.523613 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:52.524559 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:52.524738 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:52.529057 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:52.529066 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:52.529074 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:52.529126 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:52.533097 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:52.534201 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:52.534210 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:52.534217 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:52.534404 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:52.535068 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:52.535076 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:52.535081 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:52.535487 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:52.535665 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:52.536641 (+ 976us) server_negotiation.cc:299] Negotiation successful
0923 20:46:52.536922 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":1060,"threads_started":1}
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:52.538185 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:52.513062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41997 (local address 127.5.71.130:42021)
0923 20:46:52.514430 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:52.514489 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:52.517234 (+ 2745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:52.518191 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:52.518204 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:52.519095 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:52.519948 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:52.519978 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:52.521430 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:52.521438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:52.523410 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:52.523424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:52.524890 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:52.524896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:52.525128 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:52.525976 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:52.526006 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:52.528724 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:52.533354 (+ 4630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:52.533364 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:52.533382 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:52.533967 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:52.534590 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:52.534599 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:52.534602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:52.534919 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:52.535635 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:52.535642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:52.536212 (+ 570us) client_negotiation.cc:769] Sending connection context
0923 20:46:52.536480 (+ 268us) client_negotiation.cc:240] Negotiation successful
0923 20:46:52.536926 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":751,"threads_started":1}
I20260923 20:46:52.540346 11385 heartbeater.cc:347] Connected to a master server at 127.5.71.190:41997
I20260923 20:46:52.540701 11385 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:46:52.541695 11385 heartbeater.cc:511] Master 127.5.71.190:41997 requested a full tablet report, sending...
I20260923 20:46:52.543473 11062 ts_manager.cc:194] Registered new tserver with Master: 87616014d92144c1a58befa648bb5584 (127.5.71.130:42977)
I20260923 20:46:52.544525 11062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:42021
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:46:52.572919 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41997
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:52.921350 11391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:52.921695 11391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:52.921856 11391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:52.929525 11391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:52.929685 11391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:52.929859 11391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:52.940860 11391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:52.942193 11391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:52.943761 11391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:52.957532 11399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:52.957567 11396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:52.957589 11397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:52.958520 11391 server_base.cc:1034] running on GCE node
I20260923 20:46:52.959188 11391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:52.960254 11391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:52.961481 11391 hybrid_clock.cc:648] HybridClock initialized: now 1790196412961423 us; error 57 us; skew 500 ppm
I20260923 20:46:52.961835 11391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:52 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:52.964986 11391 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:46:52.966645 11391 webserver.cc:460] Webserver started at http://127.5.71.131:46311/ using document root <none> and password file <none>
I20260923 20:46:52.967350 11391 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:52.967491 11391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:52.967854 11391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:52.970870 11391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "d51f2bc522334529b099528603cedd76"
format_stamp: "Formatted at 2026-09-23 20:46:52 on dist-test-slave-dn29"
server_key: "f65c400514f58f4f7c848c1e214632d0"
server_key_iv: "611831150a9456cc2af490005230d936"
server_key_version: "encryptionkey@0"
I20260923 20:46:52.971661 11391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "d51f2bc522334529b099528603cedd76"
format_stamp: "Formatted at 2026-09-23 20:46:52 on dist-test-slave-dn29"
server_key: "f65c400514f58f4f7c848c1e214632d0"
server_key_iv: "611831150a9456cc2af490005230d936"
server_key_version: "encryptionkey@0"
I20260923 20:46:52.977941 11391 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:46:52.981887 11406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:52.983374 11391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:46:52.983573 11391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "d51f2bc522334529b099528603cedd76"
format_stamp: "Formatted at 2026-09-23 20:46:52 on dist-test-slave-dn29"
server_key: "f65c400514f58f4f7c848c1e214632d0"
server_key_iv: "611831150a9456cc2af490005230d936"
server_key_version: "encryptionkey@0"
I20260923 20:46:52.983843 11391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:53.004300 11251 heartbeater.cc:503] Master 127.5.71.190:41997 was elected leader, sending a full tablet report...
I20260923 20:46:53.005383 11391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:53.010125 11391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:53.010479 11391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:53.011534 11391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:53.013044 11391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:53.013151 11391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:53.013310 11391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:53.013381 11391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:53.050658 11391 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:41049
I20260923 20:46:53.050702 11518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:41049 every 8 connection(s)
I20260923 20:46:53.052243 11391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:53.056139 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11391
I20260923 20:46:53.056296 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:53.056761 5406 external_mini_cluster.cc:1428] Setting key dc766a2f3edfa56556aea6340b6c18fa
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196412, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:46:53.076570 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.055422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:59529 (local address 127.5.71.190:41997)
0923 20:46:53.056073 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.056081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.057426 (+ 1345us) server_negotiation.cc:407] Connection header received
0923 20:46:53.059666 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.059671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.059766 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.059920 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.061800 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.062596 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.064665 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.065156 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.065302 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.069594 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.069600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.069604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.069730 (+ 126us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.073275 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.074136 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.074141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.074143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.074281 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.074706 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.074709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.074711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.074976 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.075104 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.076009 (+ 905us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.076213 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":317,"threads_started":1}
I20260923 20:46:53.077539 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.056112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41997 (local address 127.5.71.131:59529)
0923 20:46:53.057184 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:53.057257 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:53.059388 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:53.060300 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:53.060316 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:53.060965 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:53.061607 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:53.061635 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.062782 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.062787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:53.064459 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:53.064475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.065439 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.065446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:53.065784 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.066531 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:53.066562 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:53.069319 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:53.073490 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:53.073501 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:53.073517 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:53.074001 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:53.074397 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:53.074401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:53.074403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:53.074595 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:53.075116 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:53.075124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:53.075529 (+ 405us) client_negotiation.cc:769] Sending connection context
0923 20:46:53.075890 (+ 361us) client_negotiation.cc:240] Negotiation successful
0923 20:46:53.076335 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":429,"threads_started":1}
I20260923 20:46:53.079322 11519 heartbeater.cc:347] Connected to a master server at 127.5.71.190:41997
I20260923 20:46:53.079685 11519 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:53.080564 11519 heartbeater.cc:511] Master 127.5.71.190:41997 requested a full tablet report, sending...
I20260923 20:46:53.082111 11062 ts_manager.cc:194] Registered new tserver with Master: d51f2bc522334529b099528603cedd76 (127.5.71.131:41049)
I20260923 20:46:53.083039 11062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:59529
I20260923 20:46:53.086047 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/test-admin.keytab.
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.113888 5406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 20:46:53.119483 11527 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:53.116818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49862 (local address 127.5.71.129:34121)
0923 20:46:53.117498 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.117505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.117531 (+ 26us) server_negotiation.cc:407] Connection header received
0923 20:46:53.117592 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.117599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.117888 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.118019 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.118996 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49862: BlockingRecv error: recv got EOF from 127.0.0.1:49862 (error 108)
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":354,"threads_started":1}
W20260923 20:46:53.119555 5406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc' of type FILE
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.121541 5406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.123140 5406 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:53.141209 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.124962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49866 (local address 127.5.71.129:34121)
0923 20:46:53.125126 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.125131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.125449 (+ 318us) server_negotiation.cc:407] Connection header received
0923 20:46:53.125747 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.125752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.125853 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.125949 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.127526 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.128642 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.129897 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.130443 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.130544 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.134627 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.134649 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.134674 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.134721 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.138113 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.139004 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.139012 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.139015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.139145 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.139581 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.139585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.139587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.140155 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.140408 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.140738 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.140957 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260923 20:46:53.148037 11527 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:53.146045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.5.71.129:34121)
0923 20:46:53.146287 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.146294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.146520 (+ 226us) server_negotiation.cc:407] Connection header received
0923 20:46:53.146723 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.146726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.146796 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.146896 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.147738 (+ 842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49874: BlockingRecv error: recv got EOF from 127.0.0.1:49874 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260923 20:46:53.148300 5406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc' of type FILE
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.150002 5406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.151512 5406 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:53.167027 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.153048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.5.71.129:34121)
0923 20:46:53.153283 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.153289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.153507 (+ 218us) server_negotiation.cc:407] Connection header received
0923 20:46:53.153636 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.153640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.153699 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.153800 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.154839 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.155566 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.156966 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.157375 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.157477 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.161568 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.161574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.161577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.161622 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.164218 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.165003 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.165007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.165010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.165137 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.165628 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.165632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.165634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.165941 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.166137 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.166426 (+ 289us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.166586 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260923 20:46:53.171370 11527 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:46:53.169776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.5.71.129:34121)
0923 20:46:53.170013 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.170018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.170216 (+ 198us) server_negotiation.cc:407] Connection header received
0923 20:46:53.170321 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.170325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.170394 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.170470 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.171092 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49894: BlockingRecv error: recv got EOF from 127.0.0.1:49894 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260923 20:46:53.171451 5406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790196359367181-5406-0/krb5kdc/krb5cc' of type FILE
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.173115 5406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:53.174420 5406 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:46:53.189733 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.175657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.5.71.129:34121)
0923 20:46:53.175925 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.175933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.176156 (+ 223us) server_negotiation.cc:407] Connection header received
0923 20:46:53.176386 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.176391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.176446 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.176545 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.177465 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.178191 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.179505 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.179977 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.180102 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.184415 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.184420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.184423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.184458 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.186988 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.187965 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.187969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.187971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.188156 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.188552 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.188556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.188558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.188871 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.189176 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.189306 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.189472 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 20:46:53.191497 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11123
I20260923 20:46:53.205376 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.192439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.5.71.129:34121)
0923 20:46:53.192648 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.192655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.193033 (+ 378us) server_negotiation.cc:407] Connection header received
0923 20:46:53.193099 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.193102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.193166 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.193256 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.194290 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.195058 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.196404 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.196883 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.197005 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.199285 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.199289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.199293 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.199325 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.201917 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.202636 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.202640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.202642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.202847 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.203363 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.203371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.203377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.203944 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.204196 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.204733 (+ 537us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.204957 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 20:46:53.206012 11246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:53.292222 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11123
I20260923 20:46:53.309516 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11257
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:46:53.328020 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.310504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.5.71.130:42977)
0923 20:46:53.311086 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.311096 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.311128 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:46:53.311211 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.311220 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.311528 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.311845 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.313047 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.314655 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.316192 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.317102 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.317235 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.321570 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.321585 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.321603 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.321643 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.324941 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.325742 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.325752 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.325754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.325882 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.326292 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.326296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.326298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.326662 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.326982 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.327370 (+ 388us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.327624 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":248,"threads_started":1}
I20260923 20:46:53.329408 11380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:53.414027 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11257
I20260923 20:46:53.429993 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11391
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:46:53.448096 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:53.431242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.5.71.131:41049)
0923 20:46:53.432027 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:53.432034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:53.432064 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:46:53.432135 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:53.432142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:53.432414 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:53.432605 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:53.433703 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.434748 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.436097 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:53.436709 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:53.437051 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:53.441370 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:53.441385 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:53.441403 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:53.441449 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:53.444751 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.445676 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.445685 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.445688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.445825 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:53.446221 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:53.446236 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:53.446243 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:53.446598 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:53.446924 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:53.447240 (+ 316us) server_negotiation.cc:299] Negotiation successful
0923 20:46:53.447522 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":408,"threads_started":1}
I20260923 20:46:53.449604 11514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:53.538316 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11391
I20260923 20:46:53.554225 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11030
I20260923 20:46:53.555533 11092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:53.659209 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11030
2026-09-23T20:46:53Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4181 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11547](info): set up 2 sockets
Sep 23 20:46:53 dist-test-slave-dn29 krb5kdc[11547](info): commencing operation
krb5kdc: starting...
W20260923 20:46:54.769769 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:46:54 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:46:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab.
Entry for principal oryx/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab.
I20260923 20:46:55.000360 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40203
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab
--principal=oryx/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:55.335707 11560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:55.336148 11560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:55.336305 11560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:55.343698 11560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:55.343894 11560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:55.344002 11560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:55.344110 11560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:55.344187 11560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:55.354437 11560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab
--principal=oryx/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:55.356009 11560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:55.357409 11560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:55.369778 11566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:55.369975 11568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:55.370018 11565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:55.371393 11560 server_base.cc:1034] running on GCE node
I20260923 20:46:55.372268 11560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:55.373879 11560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:55.375177 11560 hybrid_clock.cc:648] HybridClock initialized: now 1790196415375090 us; error 51 us; skew 500 ppm
I20260923 20:46:55.375641 11560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:55 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196415, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:55.379428 11560 init.cc:377] Logged in from keytab as oryx/127.5.71.190@KRBTEST.COM (short username oryx)
I20260923 20:46:55.381203 11560 webserver.cc:460] Webserver started at http://127.5.71.190:45677/ using document root <none> and password file <none>
I20260923 20:46:55.381958 11560 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:55.382069 11560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:55.382344 11560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:55.385823 11560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "ac3d81454f8c428d9d0567c0acb9b38e"
format_stamp: "Formatted at 2026-09-23 20:46:55 on dist-test-slave-dn29"
server_key: "d81bf1db995fc23a13e9282048c626f5"
server_key_iv: "28fe6752c6096f16b9b516fb8aaac8ac"
server_key_version: "encryptionkey@0"
I20260923 20:46:55.386626 11560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "ac3d81454f8c428d9d0567c0acb9b38e"
format_stamp: "Formatted at 2026-09-23 20:46:55 on dist-test-slave-dn29"
server_key: "d81bf1db995fc23a13e9282048c626f5"
server_key_iv: "28fe6752c6096f16b9b516fb8aaac8ac"
server_key_version: "encryptionkey@0"
I20260923 20:46:55.392728 11560 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:46:55.396873 11575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:55.398449 11560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260923 20:46:55.398604 11560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "ac3d81454f8c428d9d0567c0acb9b38e"
format_stamp: "Formatted at 2026-09-23 20:46:55 on dist-test-slave-dn29"
server_key: "d81bf1db995fc23a13e9282048c626f5"
server_key_iv: "28fe6752c6096f16b9b516fb8aaac8ac"
server_key_version: "encryptionkey@0"
I20260923 20:46:55.398783 11560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:55.411185 11560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:55.415524 11560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:55.415766 11560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:55.435467 11560 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:40203
I20260923 20:46:55.435487 11626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:40203 every 8 connection(s)
I20260923 20:46:55.437012 11560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:46:55.440982 11627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:55.441890 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11560
I20260923 20:46:55.442111 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:46:55.442960 5406 external_mini_cluster.cc:1428] Setting key f231dbf1b375e81039c3020a62ec0cdf
I20260923 20:46:55.454269 11627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Bootstrap starting.
Sep 23 20:46:55 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:46:55.457896 11627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:55.459101 11627 log.cc:824] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:55.462594 11627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: No bootstrap required, opened a new log
I20260923 20:46:55.466382 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:55.445265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.5.71.190:40203)
0923 20:46:55.446073 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:55.446085 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:55.446127 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:46:55.447926 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:55.447952 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:55.448400 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:55.448827 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:55.450299 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:55.451658 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:55.452992 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:55.453688 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:55.453921 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:55.458310 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:55.458320 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:55.458341 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:55.458382 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:55.461948 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:55.462701 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:55.462717 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:55.462752 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:55.462942 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:55.463359 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:55.463364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:55.463367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:55.464001 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:55.464355 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:55.464820 (+ 465us) server_negotiation.cc:299] Negotiation successful
0923 20:46:55.465296 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":350,"threads_started":1}
I20260923 20:46:55.471418 11627 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } }
I20260923 20:46:55.471863 11627 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:55.472010 11627 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3d81454f8c428d9d0567c0acb9b38e, State: Initialized, Role: FOLLOWER
I20260923 20:46:55.472645 11627 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } }
I20260923 20:46:55.472918 11627 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:46:55.473058 11627 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:46:55.473238 11627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:55.475744 11627 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } }
I20260923 20:46:55.476321 11627 leader_election.cc:302] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3d81454f8c428d9d0567c0acb9b38e; no voters:
I20260923 20:46:55.477012 11627 leader_election.cc:288] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:46:55.477159 11632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:46:55.477952 11632 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 LEADER]: Becoming Leader. State: Replica: ac3d81454f8c428d9d0567c0acb9b38e, State: Running, Role: LEADER
I20260923 20:46:55.478570 11632 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } }
I20260923 20:46:55.479297 11627 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:55.481848 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } }
I20260923 20:46:55.482296 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:55.482532 11634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3d81454f8c428d9d0567c0acb9b38e. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } }
I20260923 20:46:55.482911 11634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:55.485004 11640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:46:55.491307 11640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:46:55.501363 11640 catalog_manager.cc:1347] Generated new cluster ID: 667c948b58ec4820b40940210c9013a2
I20260923 20:46:55.501528 11640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:46:55.539772 11640 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:46:55.541409 11640 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:46:55.555831 11640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Generated new TSK 0
I20260923 20:46:55.556835 11640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab.
Entry for principal oryx/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab.
I20260923 20:46:55.616755 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40203
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:55.952989 11653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:55.953291 11653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:55.953388 11653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:55.960690 11653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:55.960819 11653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:55.961002 11653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:55.971354 11653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:55.972774 11653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:55.974119 11653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:55.989370 11661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:55.989445 11658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:55.989358 11659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:55.993942 11653 server_base.cc:1034] running on GCE node
I20260923 20:46:55.994724 11653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:55.995852 11653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:55.997128 11653 hybrid_clock.cc:648] HybridClock initialized: now 1790196415997065 us; error 52 us; skew 500 ppm
I20260923 20:46:55.997466 11653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:55 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196415, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:56.001171 11653 init.cc:377] Logged in from keytab as oryx/127.5.71.129@KRBTEST.COM (short username oryx)
I20260923 20:46:56.002952 11653 webserver.cc:460] Webserver started at http://127.5.71.129:33965/ using document root <none> and password file <none>
I20260923 20:46:56.003844 11653 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:56.003976 11653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:56.004288 11653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:56.007403 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "c8b4262b58f540f2bb5432a28e9e2c3b"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "854d5d71d6afdc6329c2848796832162"
server_key_iv: "eddaab1e0dae79659938a57dc986161f"
server_key_version: "encryptionkey@0"
I20260923 20:46:56.008262 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "c8b4262b58f540f2bb5432a28e9e2c3b"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "854d5d71d6afdc6329c2848796832162"
server_key_iv: "eddaab1e0dae79659938a57dc986161f"
server_key_version: "encryptionkey@0"
I20260923 20:46:56.014287 11653 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260923 20:46:56.018399 11668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:56.020107 11653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:56.020365 11653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "c8b4262b58f540f2bb5432a28e9e2c3b"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "854d5d71d6afdc6329c2848796832162"
server_key_iv: "eddaab1e0dae79659938a57dc986161f"
server_key_version: "encryptionkey@0"
I20260923 20:46:56.020579 11653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:56.048499 11653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:56.053718 11653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:56.054066 11653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:56.055306 11653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:56.056984 11653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:56.057066 11653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:56.057251 11653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:56.057317 11653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:56.096030 11653 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:36811
I20260923 20:46:56.096088 11780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:36811 every 8 connection(s)
I20260923 20:46:56.097464 11653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:56.099625 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11653
I20260923 20:46:56.099890 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:46:56.100889 5406 external_mini_cluster.cc:1428] Setting key af67775bfc85f64903e8aeadbca90b48
Sep 23 20:46:56 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196415, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:46:56.121322 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:56.100780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:56765 (local address 127.5.71.190:40203)
0923 20:46:56.101396 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:56.101401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:56.102651 (+ 1250us) server_negotiation.cc:407] Connection header received
0923 20:46:56.104833 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:56.104843 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:56.104945 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:56.105078 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:56.106717 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.108053 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.109722 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.110113 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.110260 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:56.114878 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:56.114884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:56.114888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:56.114924 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:56.117670 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:56.118707 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:56.118714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:56.118719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:56.118882 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:56.119412 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:56.119415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:56.119417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:56.119693 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:56.119918 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:56.120862 (+ 944us) server_negotiation.cc:299] Negotiation successful
0923 20:46:56.121033 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":360,"threads_started":1}
I20260923 20:46:56.122988 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:56.101516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:56765)
0923 20:46:56.102472 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:56.102521 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:56.104536 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:56.105341 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:56.105350 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:56.105896 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:56.106520 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:56.106539 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.108222 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.108226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:56.109475 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:56.109487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.110348 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.110355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:56.110694 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:56.111456 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:56.111491 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:56.114592 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:56.117825 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:56.117838 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:56.117860 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:56.118547 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:56.119011 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:56.119018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:56.119024 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:56.119297 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:56.119851 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:56.119862 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:56.120393 (+ 531us) client_negotiation.cc:769] Sending connection context
0923 20:46:56.120750 (+ 357us) client_negotiation.cc:240] Negotiation successful
0923 20:46:56.121372 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":411,"threads_started":1}
WARNING: no policy specified for oryx/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:46:56.124970 11781 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40203
I20260923 20:46:56.125265 11781 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.130@KRBTEST.COM" created.
I20260923 20:46:56.126125 11781 heartbeater.cc:511] Master 127.5.71.190:40203 requested a full tablet report, sending...
I20260923 20:46:56.128475 11592 ts_manager.cc:194] Registered new tserver with Master: c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811)
I20260923 20:46:56.129963 11592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:56765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab.
Entry for principal oryx/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab.
I20260923 20:46:56.155314 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40203
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:56.490144 11787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:56.490450 11787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:56.490546 11787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:56.498366 11787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:56.498482 11787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:56.498595 11787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:56.508908 11787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:56.510227 11787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:56.512032 11787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:56.528020 11795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:56.527978 11793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:56.527995 11792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:56.528812 11787 server_base.cc:1034] running on GCE node
I20260923 20:46:56.529495 11787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:56.530565 11787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:56.531860 11787 hybrid_clock.cc:648] HybridClock initialized: now 1790196416531753 us; error 49 us; skew 500 ppm
I20260923 20:46:56.532229 11787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:56 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196416, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:56.542248 11787 init.cc:377] Logged in from keytab as oryx/127.5.71.130@KRBTEST.COM (short username oryx)
I20260923 20:46:56.544107 11787 webserver.cc:460] Webserver started at http://127.5.71.130:43323/ using document root <none> and password file <none>
I20260923 20:46:56.544842 11787 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:56.544943 11787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:56.545259 11787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:56.548450 11787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "67123dd3363a4d7db208c7dd58914326"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "3aa22b9a3d6a28d0370b0d8489fb32b2"
server_key_iv: "0b4dccb6d03ef98bc4cd9eba252e8a19"
server_key_version: "encryptionkey@0"
I20260923 20:46:56.549233 11787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "67123dd3363a4d7db208c7dd58914326"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "3aa22b9a3d6a28d0370b0d8489fb32b2"
server_key_iv: "0b4dccb6d03ef98bc4cd9eba252e8a19"
server_key_version: "encryptionkey@0"
I20260923 20:46:56.554585 11787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:56.558754 11802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:56.560765 11787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260923 20:46:56.560957 11787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "67123dd3363a4d7db208c7dd58914326"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "3aa22b9a3d6a28d0370b0d8489fb32b2"
server_key_iv: "0b4dccb6d03ef98bc4cd9eba252e8a19"
server_key_version: "encryptionkey@0"
I20260923 20:46:56.561183 11787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:56.575356 11787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:56.580257 11787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:56.580593 11787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:56.581568 11787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:56.582928 11787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:56.583016 11787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:56.583143 11787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:56.583209 11787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:56.619632 11787 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:40759
I20260923 20:46:56.619663 11914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:40759 every 8 connection(s)
I20260923 20:46:56.621089 11787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:56.627074 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11787
I20260923 20:46:56.627290 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:46:56.627775 5406 external_mini_cluster.cc:1428] Setting key 108801b0174002fa1d2127aea3d11898
Sep 23 20:46:56 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196416, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:46:56.642791 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:56.623908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:60945 (local address 127.5.71.190:40203)
0923 20:46:56.624392 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:56.624397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:56.625548 (+ 1151us) server_negotiation.cc:407] Connection header received
0923 20:46:56.627552 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:56.627558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:56.627653 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:56.627772 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:56.629492 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.630507 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.632383 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.633122 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.633260 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:56.636937 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:56.636942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:56.636945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:56.636977 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:56.639588 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:56.640376 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:56.640388 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:56.640391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:56.640515 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:56.640996 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:56.641001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:56.641004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:56.641273 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:56.641427 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:56.642138 (+ 711us) server_negotiation.cc:299] Negotiation successful
0923 20:46:56.642402 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":233,"threads_started":1}
I20260923 20:46:56.643836 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:56.624565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:60945)
0923 20:46:56.625392 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:56.625432 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:56.627187 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:56.628051 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:56.628059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:56.628582 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:56.629326 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:56.629345 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.630701 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.630705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:56.632200 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:56.632213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:56.633372 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:56.633377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:56.633595 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:56.634317 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:56.634343 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:56.636687 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:56.639753 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:56.639762 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:56.639780 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:56.640250 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:56.640677 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:56.640682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:56.640686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:56.640877 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:56.641402 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:56.641410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:56.641780 (+ 370us) client_negotiation.cc:769] Sending connection context
0923 20:46:56.642042 (+ 262us) client_negotiation.cc:240] Negotiation successful
0923 20:46:56.642622 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":289,"threads_started":1}
I20260923 20:46:56.645448 11915 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40203
I20260923 20:46:56.645789 11915 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:56.646611 11915 heartbeater.cc:511] Master 127.5.71.190:40203 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:46:56.648226 11592 ts_manager.cc:194] Registered new tserver with Master: 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:46:56.649061 11592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:60945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab.
Entry for principal oryx/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab.
I20260923 20:46:56.680222 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40203
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:57.011912 11921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:57.012269 11921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:57.012446 11921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:57.019922 11921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:57.020112 11921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:57.020313 11921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:46:57.030750 11921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:57.032182 11921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:57.033607 11921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:57.048141 11926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:57.048622 11929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:57.048645 11927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:57.053452 11921 server_base.cc:1034] running on GCE node
I20260923 20:46:57.054234 11921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:57.055274 11921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:57.056545 11921 hybrid_clock.cc:648] HybridClock initialized: now 1790196417056490 us; error 45 us; skew 500 ppm
I20260923 20:46:57.056988 11921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:57.060480 11921 init.cc:377] Logged in from keytab as oryx/127.5.71.131@KRBTEST.COM (short username oryx)
I20260923 20:46:57.062445 11921 webserver.cc:460] Webserver started at http://127.5.71.131:34115/ using document root <none> and password file <none>
I20260923 20:46:57.063268 11921 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:57.063455 11921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:57.063905 11921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:57.067370 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "557016df63df4b268775a1ab2bc4d2cc"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "e30a22cf22772be4f2afefa8e13a5f9c"
server_key_iv: "10dd26709bbfd69381bca57e5f86e385"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.068328 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "557016df63df4b268775a1ab2bc4d2cc"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "e30a22cf22772be4f2afefa8e13a5f9c"
server_key_iv: "10dd26709bbfd69381bca57e5f86e385"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.074877 11921 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260923 20:46:57.078845 11936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:57.080638 11921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:57.080965 11921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "557016df63df4b268775a1ab2bc4d2cc"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "e30a22cf22772be4f2afefa8e13a5f9c"
server_key_iv: "10dd26709bbfd69381bca57e5f86e385"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.081207 11921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:57.115110 11921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:57.119760 11921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:57.120124 11921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:57.121172 11921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:57.122784 11921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:57.122910 11921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260923 20:46:57.123013 11921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:57.123057 11921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:57.133960 11781 heartbeater.cc:503] Master 127.5.71.190:40203 was elected leader, sending a full tablet report...
I20260923 20:46:57.158318 11921 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:46615
I20260923 20:46:57.158443 12048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:46615 every 8 connection(s)
I20260923 20:46:57.159727 11921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:46:57.163496 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 11921
I20260923 20:46:57.163681 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:46:57.164342 5406 external_mini_cluster.cc:1428] Setting key c92008e5085d01ced885c582cb1075b6
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:46:57.180636 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.162403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:49261 (local address 127.5.71.190:40203)
0923 20:46:57.162895 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.162901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.164010 (+ 1109us) server_negotiation.cc:407] Connection header received
0923 20:46:57.166375 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.166381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.166465 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.166598 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.168381 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.169143 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.170684 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.171081 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.171235 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.174692 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.174699 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.174704 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.174745 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.177461 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.178222 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.178226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.178228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.178358 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.178806 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.178811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.178813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.179122 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.179291 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.180022 (+ 731us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.180242 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":223,"threads_started":1}
I20260923 20:46:57.181742 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.162881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:49261)
0923 20:46:57.163853 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.163897 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.166071 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.166854 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.166863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.167554 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.168206 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.168224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.169314 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.169318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.170547 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.170558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.171323 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.171330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.171572 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.172293 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.172314 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.174486 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.177638 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.177647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.177671 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.178075 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.178474 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.178479 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.178483 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.178674 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.179257 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.179265 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.179615 (+ 350us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.179935 (+ 320us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.180518 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":376,"threads_started":1}
I20260923 20:46:57.183578 12049 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40203
I20260923 20:46:57.183952 12049 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:57.184828 12049 heartbeater.cc:511] Master 127.5.71.190:40203 requested a full tablet report, sending...
I20260923 20:46:57.186480 11592 ts_manager.cc:194] Registered new tserver with Master: 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615)
I20260923 20:46:57.187392 11592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:49261
I20260923 20:46:57.192400 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.189.keytab.
Entry for principal oryx/127.5.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.189.keytab.
I20260923 20:46:57.245608 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:35309
--webserver_interface=127.5.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.189.keytab
--principal=oryx/127.5.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40203,127.5.71.189:35309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:57.588375 12055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:57.588676 12055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:57.588768 12055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:57.596220 12055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:57.596330 12055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:57.596377 12055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:57.596426 12055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:57.596469 12055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:57.606668 12055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40203,127.5.71.189:35309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.189.keytab
--principal=oryx/127.5.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:35309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
W20260923 20:46:57.607024 12055 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.71.190:40203,127.5.71.189:35309'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 20:46:57.608202 12055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:57.609524 12055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:57.621466 12061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:57.621444 12060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:57.621549 12063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:57.623456 12055 server_base.cc:1034] running on GCE node
I20260923 20:46:57.624447 12055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:57.626173 12055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:57.627393 12055 hybrid_clock.cc:648] HybridClock initialized: now 1790196417627332 us; error 34 us; skew 500 ppm
I20260923 20:46:57.627848 12055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:57.631042 12055 init.cc:377] Logged in from keytab as oryx/127.5.71.189@KRBTEST.COM (short username oryx)
I20260923 20:46:57.632925 12055 webserver.cc:460] Webserver started at http://127.5.71.189:44613/ using document root <none> and password file <none>
I20260923 20:46:57.633769 12055 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:57.633915 12055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:57.634270 12055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:57.637521 12055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/instance:
uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "3d400291e46307f61e05628bd4c995fd"
server_key_iv: "4aa1201c9ca167ef1a21e1b23a0ee355"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.638403 12055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal/instance:
uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "3d400291e46307f61e05628bd4c995fd"
server_key_iv: "4aa1201c9ca167ef1a21e1b23a0ee355"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.644269 12055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260923 20:46:57.647957 12070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:57.649437 12055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:57.649641 12055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "3d400291e46307f61e05628bd4c995fd"
server_key_iv: "4aa1201c9ca167ef1a21e1b23a0ee355"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.649866 12055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:57.652439 11915 heartbeater.cc:503] Master 127.5.71.190:40203 was elected leader, sending a full tablet report...
I20260923 20:46:57.670284 12055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:57.674724 12055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:57.674960 12055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:57.694155 12055 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.189:35309
I20260923 20:46:57.694205 12121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.189:35309 every 8 connection(s)
I20260923 20:46:57.695637 12055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
I20260923 20:46:57.697433 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 12055
I20260923 20:46:57.700373 12122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:57.707935 12122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.189@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:46:57.721767 12125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.699595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.5.71.189:35309)
0923 20:46:57.700670 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.700687 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.700731 (+ 44us) server_negotiation.cc:407] Connection header received
0923 20:46:57.702460 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.702487 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.702986 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.703428 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.705013 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.706390 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.707652 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.708451 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.708684 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.713834 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.713845 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.713859 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.713912 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.717401 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.718270 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.718286 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.718296 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.718476 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.718928 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.718931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.718934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.719459 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.719774 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.720325 (+ 551us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.720739 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":438,"threads_started":1}
I20260923 20:46:57.727064 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.710691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.5.71.190:40203)
0923 20:46:57.711311 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.711317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.711934 (+ 617us) server_negotiation.cc:407] Connection header received
0923 20:46:57.712075 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.712081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.712159 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.712291 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.713818 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.714918 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.716646 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.717329 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.717486 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.721149 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.721155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.721158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.721202 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.724052 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.724997 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.725002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.725004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.725128 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.725501 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.725510 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.725512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.725886 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.726036 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.726483 (+ 447us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.726704 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":295,"threads_started":1}
I20260923 20:46:57.727118 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.710680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43238)
0923 20:46:57.711681 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.711733 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.711935 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.712341 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.712345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.713246 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.713594 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.713611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.715071 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.715075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.716486 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.716498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.717564 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.717571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.717765 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.718501 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.718518 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.720952 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.724186 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.724203 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.724210 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.724864 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.725242 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.725248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.725252 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.725374 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.726029 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.726037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.726205 (+ 168us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.726381 (+ 176us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.726704 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":551,"threads_started":1}
I20260923 20:46:57.728776 12122 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196417, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.189@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
I20260923 20:46:57.740943 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.729394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58442)
0923 20:46:57.729581 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.729601 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.729736 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.730025 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.730028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.730326 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.730449 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.730456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.731451 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.731455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.732449 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.732460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.733190 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.733194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.733385 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.733897 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.733913 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.735886 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.738693 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.738697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.738700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.739064 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.739440 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.739444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.739446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.739576 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.740128 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.740131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.740255 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.740420 (+ 165us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.740661 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":2}
I20260923 20:46:57.741232 12125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.729441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.5.71.189:35309)
0923 20:46:57.729586 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.729590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.729694 (+ 104us) server_negotiation.cc:407] Connection header received
0923 20:46:57.729856 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.729860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.729914 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.729994 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.730577 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.731296 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.732598 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.733033 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.733147 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.736041 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.736046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.736049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.736077 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.738568 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.739167 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.739170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.739172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.739301 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.739681 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.739684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.739686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.740015 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.740126 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.740514 (+ 388us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.740857 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":47}
I20260923 20:46:57.746001 12122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Bootstrap starting.
I20260923 20:46:57.749135 12122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:57.750120 12122 log.cc:824] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:57.752897 12122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: No bootstrap required, opened a new log
I20260923 20:46:57.760152 12122 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } }
I20260923 20:46:57.760483 12122 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:57.760556 12122 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe3239d3f6d4b65816c1217a51df7d4, State: Initialized, Role: FOLLOWER
I20260923 20:46:57.761071 12122 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } }
I20260923 20:46:57.761885 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } }
I20260923 20:46:57.762228 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:46:57.762940 12122 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: configured and running, proceeding with master startup.
W20260923 20:46:57.772773 12140 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:46:57.772923 12140 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:46:57.775401 12055 master_runner.cc:428] Detected that this master dfe3239d3f6d4b65816c1217a51df7d4 is joining an existing cluster
I20260923 20:46:57.775550 12055 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.71.189:35309
I20260923 20:46:57.799268 12125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.781289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.5.71.189:35309)
0923 20:46:57.781463 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.781469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.783458 (+ 1989us) server_negotiation.cc:407] Connection header received
0923 20:46:57.784055 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.784061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.784129 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.784203 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.785804 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.787018 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.789012 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.789752 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.789844 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.792431 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.792437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.792442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.792489 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.796274 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.797058 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.797066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.797072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.797213 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.797619 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.797624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.797627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.797956 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.798105 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.798662 (+ 557us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.798883 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":57}
I20260923 20:46:57.800555 12146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.780733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58444)
0923 20:46:57.783357 (+ 2624us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.783383 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.783898 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.784405 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.784410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.785143 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.785664 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.785677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.787230 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.787235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.788840 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.788859 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.790134 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.790140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.790346 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.791097 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.791123 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.792242 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.796422 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.796427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.796432 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.796919 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.797350 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.797354 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.797358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.797508 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.798172 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.798176 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.798389 (+ 213us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.799825 (+ 1436us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.800102 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2469,"thread_start_us":2307,"threads_started":1}
I20260923 20:46:57.804888 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.780655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.5.71.190:40203)
0923 20:46:57.780879 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.780887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.784168 (+ 3281us) server_negotiation.cc:407] Connection header received
0923 20:46:57.784479 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.784489 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.784599 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.784680 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.787387 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.788664 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.791656 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.792622 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.792741 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.796753 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.796761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.796769 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.796822 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.801107 (+ 4285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.802044 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.802052 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.802055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.802227 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.802829 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.802865 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.802872 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.803299 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.803449 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.804146 (+ 697us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.804438 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260923 20:46:57.806507 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.781637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43246)
0923 20:46:57.784009 (+ 2372us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.784030 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.784301 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.785004 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.785009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.785863 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.787164 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.787180 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.788889 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.788898 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.791462 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.791484 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.793061 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.793071 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.793670 (+ 599us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.794636 (+ 966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.794665 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.796493 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.801316 (+ 4823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.801325 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.801333 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.801891 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.802454 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.802462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.802470 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.802679 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.803558 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.803566 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.803818 (+ 252us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.805399 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.805797 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2213,"mutex_wait_us":778,"thread_start_us":1951,"threads_started":1}
WARNING: no policy specified for oryx/127.5.71.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.188@KRBTEST.COM" created.
Sep 23 20:46:57 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196415, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.190@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
I20260923 20:46:57.834779 12125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.813825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.5.71.189:35309)
0923 20:46:57.814005 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.814012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.814454 (+ 442us) server_negotiation.cc:407] Connection header received
0923 20:46:57.814871 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.814877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.814995 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.815094 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.817834 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.818999 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.821380 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.821897 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.822085 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.826582 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.826592 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.826601 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.826654 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.831084 (+ 4430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.832152 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.832164 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.832168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.832340 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.832838 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.832844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.832847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.833265 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.833420 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.834111 (+ 691us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.834351 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 20:46:57.835028 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.813589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58460)
0923 20:46:57.814315 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.814362 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.814625 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.816237 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.816249 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.817282 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.817634 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.817650 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.819172 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.819182 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.821207 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.821227 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.822109 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.822116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.822351 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.823098 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.823130 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.826364 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.831255 (+ 4891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.831269 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.831278 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.831924 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.832484 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.832491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.832495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.832697 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.833445 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.833457 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.833720 (+ 263us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.833982 (+ 262us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.834491 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":415,"threads_started":1}
I20260923 20:46:57.837636 11592 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.71.189:35309: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
member_type: NON_VOTER
last_known_addr {
host: "127.5.71.189"
port: 35309
}
attrs {
promote: true
}
}
dest_uuid: "ac3d81454f8c428d9d0567c0acb9b38e"
cas_config_opid_index: -1
I20260923 20:46:57.838757 11592 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } }
I20260923 20:46:57.842478 12153 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:57.843487 12097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.188.keytab.
Entry for principal oryx/127.5.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.188.keytab.
I20260923 20:46:57.848279 12097 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Refusing update from remote peer ac3d81454f8c428d9d0567c0acb9b38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 20:46:57.849068 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:35943
--webserver_interface=127.5.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.188.keytab
--principal=oryx/127.5.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:46:57.849463 12152 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:57.859076 12153 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.71.189:35309
I20260923 20:46:57.861307 12055 master.cc:537] Master@127.5.71.189:35309 shutting down...
I20260923 20:46:57.861451 12152 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:57.861758 12152 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:57.892235 12097 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:57.897056 12162 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:57.897578 12162 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:57.897926 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3d81454f8c428d9d0567c0acb9b38e. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:57.898167 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:57.897980 12159 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:57.898329 12159 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:57.914239 12055 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 LEARNER]: Raft consensus shutting down.
I20260923 20:46:57.914860 12055 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 LEARNER]: Raft consensus is shut down!
I20260923 20:46:57.915062 12055 tablet_replica.cc:330] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: stopping tablet replica
I20260923 20:46:57.933779 12055 master.cc:552] Master@127.5.71.189:35309 shutdown complete.
I20260923 20:46:57.933888 12055 master_runner.cc:305] Clearing existing system tablet
I20260923 20:46:57.934429 12055 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:57.934542 12055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:57.940610 12055 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:46:57.942440 12164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:57.942852 12055 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:57.943142 12055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260923 20:46:57.943235 12055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "3d400291e46307f61e05628bd4c995fd"
server_key_iv: "4aa1201c9ca167ef1a21e1b23a0ee355"
server_key_version: "encryptionkey@0"
I20260923 20:46:57.948307 12055 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:57.956188 12055 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 20:46:57.956387 12055 log.cc:1196] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 20:46:57.956954 12055 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Deleting consensus metadata
I20260923 20:46:57.957293 12055 master_runner.cc:315] Copying system tablet from 127.5.71.190:40203
I20260923 20:46:57.960206 12055 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: tablet copy: Beginning tablet copy session from remote peer at address 127.5.71.190:40203
I20260923 20:46:57.977228 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.961027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43256)
0923 20:46:57.961609 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:57.961637 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:57.961871 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:57.962317 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:57.962322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:57.962900 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:57.963270 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.963284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.964853 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.964861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.966909 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:57.966926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.967990 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.967997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:57.968195 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.968986 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:57.969012 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:57.970116 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:57.974100 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.974106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.974110 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.974640 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.975059 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:57.975064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:57.975067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:57.975237 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:57.975998 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:57.976003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:57.976186 (+ 183us) client_negotiation.cc:769] Sending connection context
0923 20:46:57.976358 (+ 172us) client_negotiation.cc:240] Negotiation successful
0923 20:46:57.976705 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":504,"threads_started":1}
I20260923 20:46:57.977324 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:57.961332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.5.71.190:40203)
0923 20:46:57.961526 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:57.961532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:57.961745 (+ 213us) server_negotiation.cc:407] Connection header received
0923 20:46:57.962052 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:57.962060 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:57.962147 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:57.962265 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:57.963432 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.964679 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.967065 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:57.967746 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:57.967919 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:57.970379 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:57.970387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:57.970392 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:57.970434 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:57.973928 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.974769 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.974774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.974778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.974924 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:57.975402 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:57.975407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:57.975410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:57.975761 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:57.975928 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:57.976491 (+ 563us) server_negotiation.cc:299] Negotiation successful
0923 20:46:57.976810 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260923 20:46:57.978730 11612 tablet_copy_service.cc:140] P ac3d81454f8c428d9d0567c0acb9b38e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer dfe3239d3f6d4b65816c1217a51df7d4 ({username='oryx', principal='oryx/127.5.71.189@KRBTEST.COM'} at 127.0.0.1:43256)
I20260923 20:46:57.979175 11612 tablet_copy_service.cc:161] P ac3d81454f8c428d9d0567c0acb9b38e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer dfe3239d3f6d4b65816c1217a51df7d4 at {username='oryx', principal='oryx/127.5.71.189@KRBTEST.COM'} at 127.0.0.1:43256: session id = dfe3239d3f6d4b65816c1217a51df7d4-00000000000000000000000000000000
I20260923 20:46:57.984220 11612 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 20:46:57.986871 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 20:46:57.994668 12055 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: tablet copy: Starting download of 0 data blocks...
I20260923 20:46:57.995136 12055 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: tablet copy: Starting download of 1 WAL segments...
I20260923 20:46:57.998524 12055 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 20:46:58.003726 11612 tablet_copy_service.cc:342] P ac3d81454f8c428d9d0567c0acb9b38e: Request end of tablet copy session dfe3239d3f6d4b65816c1217a51df7d4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.71.189@KRBTEST.COM'} at 127.0.0.1:43256
I20260923 20:46:58.004006 11612 tablet_copy_service.cc:433] P ac3d81454f8c428d9d0567c0acb9b38e: ending tablet copy session dfe3239d3f6d4b65816c1217a51df7d4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer dfe3239d3f6d4b65816c1217a51df7d4
I20260923 20:46:58.018219 12055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:58.021145 12171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:58.021656 12172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:58.026679 12055 server_base.cc:1034] running on GCE node
W20260923 20:46:58.027230 12174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:58.027707 12055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:58.028400 12055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:58.029763 12055 hybrid_clock.cc:648] HybridClock initialized: now 1790196418029687 us; error 45 us; skew 500 ppm
I20260923 20:46:58.030009 12055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196418, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:58.032526 12055 init.cc:377] Logged in from keytab as oryx/127.5.71.189@KRBTEST.COM (short username oryx)
I20260923 20:46:58.033847 12055 webserver.cc:460] Webserver started at http://127.5.71.189:36729/ using document root <none> and password file <none>
I20260923 20:46:58.034222 12055 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:58.034320 12055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:58.038228 12055 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:58.040350 12181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:58.040796 12055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260923 20:46:58.040980 12055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "3d400291e46307f61e05628bd4c995fd"
server_key_iv: "4aa1201c9ca167ef1a21e1b23a0ee355"
server_key_version: "encryptionkey@0"
I20260923 20:46:58.041209 12055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:58.068324 12055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:58.072961 12055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:58.132486 12055 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.189:35309
I20260923 20:46:58.132512 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.189:35309 every 8 connection(s)
I20260923 20:46:58.133823 12055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
I20260923 20:46:58.140285 12233 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:46:58.145287 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Bootstrap starting.
I20260923 20:46:58.163957 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:46:58.164878 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: Bootstrap complete.
I20260923 20:46:58.166438 12233 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } }
I20260923 20:46:58.166740 12233 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dfe3239d3f6d4b65816c1217a51df7d4, State: Initialized, Role: LEARNER
I20260923 20:46:58.167073 12233 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } }
I20260923 20:46:58.168274 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.168597 12233 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:58.168668 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:58.179904 12247 catalog_manager.cc:1259] Loaded cluster ID: 667c948b58ec4820b40940210c9013a2
I20260923 20:46:58.180022 12247 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: loading cluster ID for follower catalog manager: success
I20260923 20:46:58.182613 12247 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: acquiring CA information for follower catalog manager: success
I20260923 20:46:58.185015 12247 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 20:46:58.190007 12049 heartbeater.cc:503] Master 127.5.71.190:40203 was elected leader, sending a full tablet report...
W20260923 20:46:58.335971 12156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:58.336254 12156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:58.336342 12156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:58.343909 12156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:46:58.344020 12156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:58.344064 12156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:58.344120 12156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:46:58.344162 12156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:46:58.354987 12156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.188.keytab
--principal=oryx/127.5.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:35943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:58.356429 12156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:58.357910 12156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:58.370859 12252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:58.371282 12255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:58.371546 12253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:58.372550 12156 server_base.cc:1034] running on GCE node
I20260923 20:46:58.373420 12156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:58.374960 12156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:58.376159 12156 hybrid_clock.cc:648] HybridClock initialized: now 1790196418376094 us; error 64 us; skew 500 ppm
I20260923 20:46:58.376526 12156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196418, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:58.379769 12156 init.cc:377] Logged in from keytab as oryx/127.5.71.188@KRBTEST.COM (short username oryx)
I20260923 20:46:58.381397 12156 webserver.cc:460] Webserver started at http://127.5.71.188:37161/ using document root <none> and password file <none>
I20260923 20:46:58.382058 12156 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:58.382161 12156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:58.382407 12156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:46:58.385547 12156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/instance:
uuid: "d0832bd8d4464f52b4c15575cd4a6d6a"
format_stamp: "Formatted at 2026-09-23 20:46:58 on dist-test-slave-dn29"
server_key: "364192c63dfebd2e0423235c4183f6bd"
server_key_iv: "4a4b428dea986834dee9ee898a4dcc4c"
server_key_version: "encryptionkey@0"
I20260923 20:46:58.386365 12156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal/instance:
uuid: "d0832bd8d4464f52b4c15575cd4a6d6a"
format_stamp: "Formatted at 2026-09-23 20:46:58 on dist-test-slave-dn29"
server_key: "364192c63dfebd2e0423235c4183f6bd"
server_key_iv: "4a4b428dea986834dee9ee898a4dcc4c"
server_key_version: "encryptionkey@0"
I20260923 20:46:58.392169 12156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:46:58.396107 12262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:58.397635 12156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:46:58.397874 12156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
uuid: "d0832bd8d4464f52b4c15575cd4a6d6a"
format_stamp: "Formatted at 2026-09-23 20:46:58 on dist-test-slave-dn29"
server_key: "364192c63dfebd2e0423235c4183f6bd"
server_key_iv: "4a4b428dea986834dee9ee898a4dcc4c"
server_key_version: "encryptionkey@0"
I20260923 20:46:58.398092 12156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:58.415011 12156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:58.419593 12156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:58.419986 12156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:58.440344 12156 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.188:35943
I20260923 20:46:58.440346 12313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.188:35943 every 8 connection(s)
I20260923 20:46:58.442118 12156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
I20260923 20:46:58.446589 12314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:46:58.449576 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 12156
I20260923 20:46:58.456492 12314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.188@KRBTEST.COM
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196418, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.188@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:46:58.480010 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.461227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.5.71.190:40203)
0923 20:46:58.461445 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.461452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.462299 (+ 847us) server_negotiation.cc:407] Connection header received
0923 20:46:58.462512 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.462519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.462608 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.462704 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.464225 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.465278 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.467581 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.468171 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.468339 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.472501 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.472513 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.472518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.472555 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.476408 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.477279 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.477286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.477290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.477456 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.477905 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.477912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.477917 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.478309 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.478458 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.479414 (+ 956us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.479644 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 20:46:58.480019 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.452101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.5.71.188:35943)
0923 20:46:58.456184 (+ 4083us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.456199 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.456245 (+ 46us) server_negotiation.cc:407] Connection header received
0923 20:46:58.458004 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.458031 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.458540 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.459006 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.460864 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.462486 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.464488 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.465700 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.465862 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.471494 (+ 5632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.471506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.471529 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.471565 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.474973 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.475672 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.475680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.475688 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.475901 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.476294 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.476298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.476302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.477053 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.477387 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.477993 (+ 606us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.478448 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3826,"thread_start_us":3505,"threads_started":1}
I20260923 20:46:58.480624 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.461201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43264)
0923 20:46:58.462147 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.462187 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.462380 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.462848 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.462856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.463703 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.464026 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.464039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.465486 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.465492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.467418 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.467434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.468447 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.468455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.468678 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.469389 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.469425 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.472227 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.476555 (+ 4328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.476563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.476569 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.477138 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.477575 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.477582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.477585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.477751 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.478522 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.478529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.478709 (+ 180us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.478925 (+ 216us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.479235 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":470,"threads_started":1}
I20260923 20:46:58.482430 12314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:46:58.490504 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.491014 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196418, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.188@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
I20260923 20:46:58.501396 12320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.483368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.5.71.189:35309)
0923 20:46:58.483991 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.483995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.484011 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:46:58.484062 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.484065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.484136 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.484223 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.485324 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.486458 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.488535 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.489516 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.489704 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.494570 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.494581 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.494590 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.494646 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.498638 (+ 3992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.499382 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.499388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.499392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.499533 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.499968 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.499973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.499976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.500339 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.500496 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.500888 (+ 392us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.501058 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":285,"threads_started":1}
I20260923 20:46:58.501425 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.483415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58466)
0923 20:46:58.483768 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.483816 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.483945 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.484294 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.484301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.484888 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.485136 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.485152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.486611 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.486619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.488357 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.488370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.489735 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.489740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.489920 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.490635 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.490659 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.494258 (+ 3599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.498800 (+ 4542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.498805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.498809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.499247 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.499649 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.499654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.499658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.499849 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.500467 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.500472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.500626 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.500780 (+ 154us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.501085 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260923 20:46:58.502683 12314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:46:58.504242 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.491095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.5.71.189:35309)
0923 20:46:58.495286 (+ 4191us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.495295 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.497819 (+ 2524us) server_negotiation.cc:407] Connection header received
0923 20:46:58.497939 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.497944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.498040 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.498188 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:58.498688 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.499493 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.501850 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.503168 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.503342 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.503610 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.503850 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.503987 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3962,"thread_start_us":3834,"threads_started":1}
I20260923 20:46:58.504314 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.490668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58468)
0923 20:46:58.497618 (+ 6950us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.497636 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.497818 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.498260 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.498266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.498296 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:58.498521 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.498531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.499636 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.499641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.501439 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.501458 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.503374 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.503380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.503559 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.503675 (+ 116us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.503779 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.503976 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6806,"thread_start_us":4826,"threads_started":1}
I20260923 20:46:58.505195 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3d81454f8c428d9d0567c0acb9b38e. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.505468 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:58.506927 12319 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: attempting to promote NON_VOTER dfe3239d3f6d4b65816c1217a51df7d4 to VOTER
I20260923 20:46:58.507673 12319 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } }
I20260923 20:46:58.509507 12208 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 LEARNER]: Refusing update from remote peer ac3d81454f8c428d9d0567c0acb9b38e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 20:46:58.510046 12321 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196418, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.188@KRBTEST.COM for oryx/127.5.71.188@KRBTEST.COM
I20260923 20:46:58.513586 12319 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } }
I20260923 20:46:58.514917 12208 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } }
I20260923 20:46:58.517590 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } }
I20260923 20:46:58.517859 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:58.518078 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } }
I20260923 20:46:58.518301 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:58.519369 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.503431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.0.0.1:40088)
0923 20:46:58.503643 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.503667 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.503786 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.504354 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.504361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.504813 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.504940 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.504948 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.506413 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.506417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.507286 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.507296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.508093 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.508098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.508316 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.509032 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.509047 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.512517 (+ 3470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.516511 (+ 3994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.516516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.516521 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.516976 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.517376 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.517380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.517383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.517558 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.518236 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.518296 (+ 60us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.518483 (+ 187us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.518655 (+ 172us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.518931 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260923 20:46:58.519367 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.503924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40088 (local address 127.5.71.188:35943)
0923 20:46:58.504073 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.504079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.504095 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:46:58.504140 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.504146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.504212 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.504312 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.505639 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.506296 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.507415 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.507900 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.508070 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.512756 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.512763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.512767 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.512833 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.516359 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.517106 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.517114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.517120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.517257 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.517680 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.517685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.517688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.518053 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.518190 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.518725 (+ 535us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.519059 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":59}
I20260923 20:46:58.520618 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } }
I20260923 20:46:58.520850 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:46:58.524703 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Bootstrap starting.
I20260923 20:46:58.528033 12314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Neither blocks nor log segments found. Creating new log.
I20260923 20:46:58.529055 12314 log.cc:824] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:46:58.531646 12314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: No bootstrap required, opened a new log
I20260923 20:46:58.539106 12314 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } }
I20260923 20:46:58.539498 12314 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:46:58.539644 12314 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0832bd8d4464f52b4c15575cd4a6d6a, State: Initialized, Role: FOLLOWER
I20260923 20:46:58.540369 12314 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } }
I20260923 20:46:58.541266 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } } }
I20260923 20:46:58.541636 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:46:58.542441 12314 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: configured and running, proceeding with master startup.
W20260923 20:46:58.551409 12339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:46:58.551574 12339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:46:58.556059 12156 master_runner.cc:428] Detected that this master d0832bd8d4464f52b4c15575cd4a6d6a is joining an existing cluster
I20260923 20:46:58.556187 12156 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.71.188:35943
I20260923 20:46:58.579030 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.560867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58480 (local address 127.5.71.189:35309)
0923 20:46:58.561054 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.561059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.561733 (+ 674us) server_negotiation.cc:407] Connection header received
0923 20:46:58.562013 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.562021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.562100 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.562157 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.565037 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.566183 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.568462 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.569140 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.569242 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.571994 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.572001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.572006 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.572048 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.575614 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.576512 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.576518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.576521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.576681 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.577236 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.577241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.577245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.577579 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.577712 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.578370 (+ 658us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.578637 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:46:58.580300 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.560676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58480)
0923 20:46:58.561628 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.561650 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.561853 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.562402 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.562407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.563130 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.563524 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.563537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.566436 (+ 2899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.566442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.568271 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.568286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.569535 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.569545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.569736 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.570520 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.570543 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.571743 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.575837 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.575843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.575847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.576364 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.576881 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.576886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.576890 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.577105 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.577808 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.577813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.578047 (+ 234us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.579299 (+ 1252us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.579810 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":603,"threads_started":1}
I20260923 20:46:58.586169 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.560663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.5.71.190:40203)
0923 20:46:58.560951 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.560961 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.568764 (+ 7803us) server_negotiation.cc:407] Connection header received
0923 20:46:58.569085 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.569093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.569200 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.569280 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.571224 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.572755 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.575093 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.575823 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.575923 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.578710 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.578716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.578721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.578771 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.582854 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.583669 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.583675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.583679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.583880 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.584392 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.584397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.584401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.584748 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.584912 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.585578 (+ 666us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.585816 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260923 20:46:58.587031 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11653
I20260923 20:46:58.587131 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.561371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43276)
0923 20:46:58.568614 (+ 7243us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.568640 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.568913 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.569635 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.569643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.570498 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.570983 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.571000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.573001 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.573008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.574902 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.574918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.576483 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.576488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.576701 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.577390 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.577412 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.578515 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.582996 (+ 4481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.583001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.583005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.583552 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.584056 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.584060 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.584064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.584262 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.584984 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.584989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.585206 (+ 217us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.586342 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.586683 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6757,"thread_start_us":6886,"threads_started":1}
I20260923 20:46:58.589958 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.561790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.0.0.1:40096)
0923 20:46:58.573667 (+ 11877us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.573689 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.573853 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.574274 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.574278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.574962 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.575309 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.575319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.576702 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.576707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.578467 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.578481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.579677 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.579682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.579862 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.580458 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.580481 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.582491 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.586548 (+ 4057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.586555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.586564 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.587013 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.588130 (+ 1117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.588134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.588138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.588270 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.588861 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.588865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.589037 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.589179 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.589504 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11729,"thread_start_us":11525,"threads_started":1}
I20260923 20:46:58.590883 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.562125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.5.71.188:35943)
0923 20:46:58.562340 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.562346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.573772 (+ 11426us) server_negotiation.cc:407] Connection header received
0923 20:46:58.573978 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.573984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.574050 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.574111 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.575443 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.576477 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.578612 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.579329 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.579446 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.582649 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.582655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.582659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.582694 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.586359 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.587189 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.587196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.587201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.587995 (+ 794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.588397 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.588402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.588405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.588720 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.590198 (+ 1478us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.590317 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.590532 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.129@KRBTEST.COM
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196415, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.190@KRBTEST.COM for oryx/127.5.71.188@KRBTEST.COM
I20260923 20:46:58.608981 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.593263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.0.0.1:40108)
0923 20:46:58.593489 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.593508 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.593612 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.594139 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.594146 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.594578 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.594712 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.594720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.595735 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.595742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.596976 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.596996 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.598051 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.598058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.598270 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.598916 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.598944 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.601827 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.606087 (+ 4260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.606093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.606097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.606640 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.607114 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.607120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.607123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.607301 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.607969 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.607974 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.608149 (+ 175us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.608310 (+ 161us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.608608 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260923 20:46:58.609712 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.588191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.5.71.129:36811)
0923 20:46:58.588992 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.588999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.589024 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:46:58.589079 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.589085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.589309 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.589482 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.591118 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.592950 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.595041 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.596028 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.596721 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.601679 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.601698 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.601720 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.601820 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.606234 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.607020 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.607032 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.607037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.607222 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.607662 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.607668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.607671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.608151 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.608462 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.608829 (+ 367us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.609105 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":415,"threads_started":1}
I20260923 20:46:58.610152 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.593545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40108 (local address 127.5.71.188:35943)
0923 20:46:58.593783 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.593788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.593804 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:46:58.593900 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.593905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.593989 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.594111 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.594867 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.595592 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.597139 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.597881 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.598007 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.602021 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.602027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.602031 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.602075 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.605924 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.606830 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.606836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.606839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.606984 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.607423 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.607428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.607435 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.607845 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.607977 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.609564 (+ 1587us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.609754 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 20:46:58.611351 11776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:58.611683 11592 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.71.188:35943: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a"
member_type: NON_VOTER
last_known_addr {
host: "127.5.71.188"
port: 35943
}
attrs {
promote: true
}
}
dest_uuid: "ac3d81454f8c428d9d0567c0acb9b38e"
cas_config_opid_index: 6
I20260923 20:46:58.612511 11592 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } }
I20260923 20:46:58.614778 12208 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Refusing update from remote peer ac3d81454f8c428d9d0567c0acb9b38e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 20:46:58.615167 12289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:46:58.615406 12321 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260923 20:46:58.618042 12321 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.621515 12289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Refusing update from remote peer ac3d81454f8c428d9d0567c0acb9b38e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 20:46:58.623157 12322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:46:58.623976 12328 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:46:58.624516 12328 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } }
I20260923 20:46:58.626924 12321 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.71.188:35943
I20260923 20:46:58.627576 12322 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.628069 12322 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:58.627650 12208 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.628361 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.628818 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:58.631647 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.632128 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:46:58.648491 12328 leader_election.cc:288] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203), dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189:35309)
I20260923 20:46:58.649068 12156 master.cc:537] Master@127.5.71.188:35943 shutting down...
I20260923 20:46:58.650080 11602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" is_pre_election: true
I20260923 20:46:58.654312 12208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" is_pre_election: true
I20260923 20:46:58.655599 12263 leader_election.cc:302] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0832bd8d4464f52b4c15575cd4a6d6a; no voters: ac3d81454f8c428d9d0567c0acb9b38e, dfe3239d3f6d4b65816c1217a51df7d4
I20260923 20:46:58.668239 12289 raft_consensus.cc:858] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } }
W20260923 20:46:58.668478 12289 raft_consensus.cc:1565] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } }
W20260923 20:46:58.672338 12268 tablet.cc:2369] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:46:58.681190 12289 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.690587 12353 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.690874 12353 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260923 20:46:58.691422 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3d81454f8c428d9d0567c0acb9b38e. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.691651 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260923 20:46:58.691891 12351 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 NON_PARTICIPANT]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:46:58.692591 12356 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: NON_VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: true } } }
I20260923 20:46:58.692878 12356 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260923 20:46:58.693475 12156 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260923 20:46:58.694082 12156 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260923 20:46:58.694044 12322 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.694286 12156 tablet_replica.cc:330] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: stopping tablet replica
I20260923 20:46:58.694324 12322 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
W20260923 20:46:58.695478 11576 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 20:46:58.699110 11576 connection.cc:535] client connection to 127.5.71.188:35943 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260923 20:46:58.699429 11576 consensus_peers.cc:487] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e -> Peer d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188:35943): Couldn't send request to peer d0832bd8d4464f52b4c15575cd4a6d6a. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260923 20:46:58.713881 12156 master.cc:552] Master@127.5.71.188:35943 shutdown complete.
I20260923 20:46:58.713992 12156 master_runner.cc:305] Clearing existing system tablet
I20260923 20:46:58.714429 12156 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:58.714524 12156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:58.720681 12156 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.003s
I20260923 20:46:58.722694 12359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:58.723198 12156 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:58.723465 12156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260923 20:46:58.723570 12156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
uuid: "d0832bd8d4464f52b4c15575cd4a6d6a"
format_stamp: "Formatted at 2026-09-23 20:46:58 on dist-test-slave-dn29"
server_key: "364192c63dfebd2e0423235c4183f6bd"
server_key_iv: "4a4b428dea986834dee9ee898a4dcc4c"
server_key_version: "encryptionkey@0"
I20260923 20:46:58.728379 12156 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:46:58.735745 12156 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 20:46:58.735957 12156 log.cc:1196] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 20:46:58.736481 12156 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Deleting consensus metadata
I20260923 20:46:58.736830 12156 master_runner.cc:315] Copying system tablet from 127.5.71.190:40203
I20260923 20:46:58.740411 12156 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: tablet copy: Beginning tablet copy session from remote peer at address 127.5.71.190:40203
I20260923 20:46:58.764545 12126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.741314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.5.71.190:40203)
0923 20:46:58.741646 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:58.741652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:58.742765 (+ 1113us) server_negotiation.cc:407] Connection header received
0923 20:46:58.743151 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:58.743158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:58.743277 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:58.743432 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:58.745034 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.746566 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.749152 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.750071 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.750273 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.753640 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:58.753648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:58.753654 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:58.753701 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:58.759677 (+ 5976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.761022 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.761029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.761033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.761369 (+ 336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:58.761994 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:58.762000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:58.762004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:58.762549 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:58.762742 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:58.763411 (+ 669us) server_negotiation.cc:299] Negotiation successful
0923 20:46:58.763756 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260923 20:46:58.765008 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:58.741296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43286)
0923 20:46:58.742633 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:58.742654 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:58.742978 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:58.743452 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:58.743457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:58.744324 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:58.744858 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.744872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.746774 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.746780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.748965 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:58.748981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:58.750330 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:58.750335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:58.750535 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:58.751554 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:58.751576 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:58.753343 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:58.759936 (+ 6593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.759946 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.759953 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.760843 (+ 890us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.761505 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:58.761513 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:58.761519 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:58.761824 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:58.762690 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:58.762697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:58.763072 (+ 375us) client_negotiation.cc:769] Sending connection context
0923 20:46:58.763267 (+ 195us) client_negotiation.cc:240] Negotiation successful
0923 20:46:58.764140 (+ 873us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"thread_start_us":837,"threads_started":1}
I20260923 20:46:58.765690 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11653
I20260923 20:46:58.766774 11612 tablet_copy_service.cc:140] P ac3d81454f8c428d9d0567c0acb9b38e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d0832bd8d4464f52b4c15575cd4a6d6a ({username='oryx', principal='oryx/127.5.71.188@KRBTEST.COM'} at 127.0.0.1:43286)
I20260923 20:46:58.767119 11612 tablet_copy_service.cc:161] P ac3d81454f8c428d9d0567c0acb9b38e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d0832bd8d4464f52b4c15575cd4a6d6a at {username='oryx', principal='oryx/127.5.71.188@KRBTEST.COM'} at 127.0.0.1:43286: session id = d0832bd8d4464f52b4c15575cd4a6d6a-00000000000000000000000000000000
I20260923 20:46:58.771996 11612 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 20:46:58.774403 12156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 20:46:58.782596 12156 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: tablet copy: Starting download of 0 data blocks...
I20260923 20:46:58.782915 12156 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: tablet copy: Starting download of 1 WAL segments...
I20260923 20:46:58.786121 12156 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 20:46:58.787110 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:36811
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:40203
--webserver_port=33965
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:46:58.791574 11612 tablet_copy_service.cc:342] P ac3d81454f8c428d9d0567c0acb9b38e: Request end of tablet copy session d0832bd8d4464f52b4c15575cd4a6d6a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.71.188@KRBTEST.COM'} at 127.0.0.1:43286
I20260923 20:46:58.791719 11612 tablet_copy_service.cc:433] P ac3d81454f8c428d9d0567c0acb9b38e: ending tablet copy session d0832bd8d4464f52b4c15575cd4a6d6a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d0832bd8d4464f52b4c15575cd4a6d6a
I20260923 20:46:58.806010 12156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:58.809697 12367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:58.810285 12368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:58.817463 12370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:58.822189 12156 server_base.cc:1034] running on GCE node
I20260923 20:46:58.822608 12156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:58.823314 12156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:58.824513 12156 hybrid_clock.cc:648] HybridClock initialized: now 1790196418824464 us; error 40 us; skew 500 ppm
I20260923 20:46:58.824770 12156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:58 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196418, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:58.827028 12156 init.cc:377] Logged in from keytab as oryx/127.5.71.188@KRBTEST.COM (short username oryx)
I20260923 20:46:58.832854 12156 webserver.cc:460] Webserver started at http://127.5.71.188:42001/ using document root <none> and password file <none>
I20260923 20:46:58.833223 12156 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:58.833328 12156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:58.836553 12156 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260923 20:46:58.838688 12377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:58.839309 12156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260923 20:46:58.839440 12156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
uuid: "d0832bd8d4464f52b4c15575cd4a6d6a"
format_stamp: "Formatted at 2026-09-23 20:46:58 on dist-test-slave-dn29"
server_key: "364192c63dfebd2e0423235c4183f6bd"
server_key_iv: "4a4b428dea986834dee9ee898a4dcc4c"
server_key_version: "encryptionkey@0"
I20260923 20:46:58.839637 12156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:58.871469 12156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:58.875725 12156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:58.911142 12156 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.188:35943
I20260923 20:46:58.911197 12428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.188:35943 every 8 connection(s)
I20260923 20:46:58.912330 12156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
I20260923 20:46:58.917701 12429 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:46:58.921705 12429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Bootstrap starting.
I20260923 20:46:58.937425 12429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:46:58.938007 12429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Bootstrap complete.
I20260923 20:46:58.939108 12429 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } }
I20260923 20:46:58.939455 12429 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d0832bd8d4464f52b4c15575cd4a6d6a, State: Initialized, Role: LEARNER
I20260923 20:46:58.939760 12429 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } }
I20260923 20:46:58.940784 12432 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:58.941115 12429 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:46:58.941231 12432 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:58.951408 12443 catalog_manager.cc:1259] Loaded cluster ID: 667c948b58ec4820b40940210c9013a2
I20260923 20:46:58.951538 12443 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: loading cluster ID for follower catalog manager: success
I20260923 20:46:58.955350 12443 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: acquiring CA information for follower catalog manager: success
I20260923 20:46:58.958053 12443 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 20:46:59.093325 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:59.093796 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:59.101482 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.093768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.0.0.1:40122)
0923 20:46:59.094082 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.094104 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.094281 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.095203 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.095207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.095250 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:46:59.095446 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.095455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.097488 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.097494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.098733 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.098745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.100459 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.100465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.100679 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.100733 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 20:46:59.100864 (+ 131us) client_negotiation.cc:240] Negotiation successful
0923 20:46:59.101047 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260923 20:46:59.102193 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.094123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.5.71.188:35943)
0923 20:46:59.094866 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.094871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.094888 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:46:59.094949 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.094952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.095018 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.095161 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:46:59.095612 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.097314 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.098903 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.100288 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.100440 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.100866 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:59.100923 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 20:46:59.101085 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":438,"threads_started":1}
I20260923 20:46:59.103304 12404 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260923 20:46:59.103822 12432 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3d81454f8c428d9d0567c0acb9b38e. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: NON_VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: true } } }
I20260923 20:46:59.104110 12432 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: LEARNER
I20260923 20:46:59.104615 12321 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: attempting to promote NON_VOTER d0832bd8d4464f52b4c15575cd4a6d6a to VOTER
I20260923 20:46:59.105316 12321 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } }
I20260923 20:46:59.107196 12404 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 LEARNER]: Refusing update from remote peer ac3d81454f8c428d9d0567c0acb9b38e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260923 20:46:59.107388 12208 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Refusing update from remote peer ac3d81454f8c428d9d0567c0acb9b38e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260923 20:46:59.107998 12321 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260923 20:46:59.108333 12319 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260923 20:46:59.111934 12321 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.113018 12208 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.116173 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.116613 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:59.117105 12404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.119237 12236 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.119627 12236 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:46:59.120090 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.120494 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:59.121115 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.121524 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e [sys.catalog]: This master's current role is: LEADER
I20260923 20:46:59.122287 12432 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:46:59.122689 12432 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: FOLLOWER
W20260923 20:46:59.207404 12366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:59.207734 12366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:59.207859 12366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:59.215750 12366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:59.215883 12366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:59.215997 12366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:46:59.226042 12366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:36811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=33965
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:59.227350 12366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:59.228722 12366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:59.244314 12450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:59.244261 12451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:59.244794 12453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:59.245357 12366 server_base.cc:1034] running on GCE node
I20260923 20:46:59.246126 12366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:59.247103 12366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:59.248317 12366 hybrid_clock.cc:648] HybridClock initialized: now 1790196419248260 us; error 54 us; skew 500 ppm
I20260923 20:46:59.248697 12366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:59.252413 12366 init.cc:377] Logged in from keytab as oryx/127.5.71.129@KRBTEST.COM (short username oryx)
I20260923 20:46:59.253980 12366 webserver.cc:460] Webserver started at http://127.5.71.129:33965/ using document root <none> and password file <none>
I20260923 20:46:59.254674 12366 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:59.254789 12366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:59.260326 12366 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 20:46:59.263062 12460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:59.264793 12366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:59.264981 12366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "c8b4262b58f540f2bb5432a28e9e2c3b"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "854d5d71d6afdc6329c2848796832162"
server_key_iv: "eddaab1e0dae79659938a57dc986161f"
server_key_version: "encryptionkey@0"
I20260923 20:46:59.266088 12366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:59.285324 12366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:59.290217 12366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:59.290529 12366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:59.291738 12366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:59.293251 12366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:59.293372 12366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:59.293515 12366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:59.293619 12366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:59.329000 12366 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:36811
I20260923 20:46:59.329090 12572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:36811 every 8 connection(s)
I20260923 20:46:59.330518 12366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:46:59.336642 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 12366
I20260923 20:46:59.337090 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11787
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.130@KRBTEST.COM
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.188@KRBTEST.COM
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
I20260923 20:46:59.364996 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.338385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.5.71.130:40759)
0923 20:46:59.339457 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.339464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.339492 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:46:59.339558 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.339567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.339775 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.340621 (+ 846us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.342006 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343598 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.345587 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.346443 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.351013 (+ 4570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.353004 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.353017 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.353034 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.353111 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.358267 (+ 5156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.359699 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.359710 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.359714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.360084 (+ 370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.360799 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.360807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.360811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.361441 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:59.363546 (+ 2105us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:59.363738 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 20:46:59.364137 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":747,"threads_started":1}
I20260923 20:46:59.366217 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.333779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:42467 (local address 127.5.71.190:40203)
0923 20:46:59.334257 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.334263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.335611 (+ 1348us) server_negotiation.cc:407] Connection header received
0923 20:46:59.339983 (+ 4372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.339990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.340081 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.340166 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.342395 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343570 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.346501 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.347161 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.347273 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.354574 (+ 7301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.354592 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.354701 (+ 109us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.354761 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.360926 (+ 6165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.362279 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.362287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.362293 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.362548 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.363377 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.363383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.363387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.363913 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:59.364056 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:59.365300 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0923 20:46:59.365633 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"spinlock_wait_cycles":1280,"thread_start_us":319,"threads_started":1}
I20260923 20:46:59.366240 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.333751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:48073 (local address 127.5.71.188:35943)
0923 20:46:59.334095 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.334102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.335610 (+ 1508us) server_negotiation.cc:407] Connection header received
0923 20:46:59.338834 (+ 3224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.338841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.338928 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.339025 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.342395 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343580 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.346501 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.347170 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.347263 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.354701 (+ 7438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.354710 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.354714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.354785 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.361081 (+ 6296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.362500 (+ 1419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.362509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.362515 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.362803 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.363421 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.363426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.363430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.363944 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:59.364067 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:59.365287 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0923 20:46:59.365642 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260923 20:46:59.366887 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.336366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:41533 (local address 127.5.71.189:35309)
0923 20:46:59.338016 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.338021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.338042 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:46:59.339058 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.339063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.339145 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.339227 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.342346 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343533 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.346516 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.347199 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.347302 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.355957 (+ 8655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.355969 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.355978 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.356029 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.362465 (+ 6436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.363633 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.363638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.363641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.363840 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.364512 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.364517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.364520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.365080 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:59.365205 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:59.365969 (+ 764us) server_negotiation.cc:299] Negotiation successful
0923 20:46:59.366325 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1478,"thread_start_us":1348,"threads_started":1}
I20260923 20:46:59.367324 11910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:46:59.369357 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.336135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.5.71.129:41533)
0923 20:46:59.336646 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.336664 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.338906 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.339502 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.339506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.341547 (+ 2041us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:59.342141 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.342172 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343775 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.343781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.346337 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.346354 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.347742 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.347928 (+ 186us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.348170 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.349204 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:59.349228 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:59.355513 (+ 6285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:59.362652 (+ 7139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.362658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.362664 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.363445 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.364029 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.364034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.364037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.364360 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.365288 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:59.365292 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:59.365478 (+ 186us) client_negotiation.cc:769] Sending connection context
0923 20:46:59.367108 (+ 1630us) client_negotiation.cc:240] Negotiation successful
0923 20:46:59.367382 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":255,"thread_start_us":293,"threads_started":1}
I20260923 20:46:59.369300 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.334394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.5.71.129:48073)
0923 20:46:59.335424 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.335489 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.338537 (+ 3048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.339447 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.339460 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.340868 (+ 1408us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:59.342141 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.342160 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343877 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.343882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.346320 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.346335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.347741 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.347750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.348166 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.349347 (+ 1181us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:59.349369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:59.354390 (+ 5021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:59.361251 (+ 6861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.361257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.361261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.362335 (+ 1074us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.362973 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.362977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.362994 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.363246 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.364166 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:59.364170 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:59.364752 (+ 582us) client_negotiation.cc:769] Sending connection context
0923 20:46:59.366566 (+ 1814us) client_negotiation.cc:240] Negotiation successful
0923 20:46:59.367285 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":452,"spinlock_wait_cycles":71936,"thread_start_us":462,"threads_started":1}
I20260923 20:46:59.371735 12573 heartbeater.cc:347] Connected to a master server at 127.5.71.188:35943
I20260923 20:46:59.371861 12575 heartbeater.cc:347] Connected to a master server at 127.5.71.189:35309
I20260923 20:46:59.372263 12575 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:59.372290 12573 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:59.375417 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.334394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:42467)
0923 20:46:59.335424 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.335494 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.339803 (+ 4309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.340430 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.340436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.341124 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:59.342134 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.342160 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.343776 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.343781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.346319 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.346335 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.347742 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.347750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.348172 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.349069 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:59.349110 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:59.354111 (+ 5001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:59.361121 (+ 7010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.361132 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.361155 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.362130 (+ 975us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.362733 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.362739 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.362742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.363161 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.364144 (+ 983us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:59.364153 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:59.364781 (+ 628us) client_negotiation.cc:769] Sending connection context
0923 20:46:59.366488 (+ 1707us) client_negotiation.cc:240] Negotiation successful
0923 20:46:59.372293 (+ 5805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":462,"spinlock_wait_cycles":23680,"thread_start_us":450,"threads_started":1}
I20260923 20:46:59.375859 12573 heartbeater.cc:511] Master 127.5.71.188:35943 requested a full tablet report, sending...
I20260923 20:46:59.376510 12575 heartbeater.cc:511] Master 127.5.71.189:35309 requested a full tablet report, sending...
I20260923 20:46:59.376796 12574 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40203
I20260923 20:46:59.377089 12574 heartbeater.cc:464] Registering TS with master...
I20260923 20:46:59.378124 12574 heartbeater.cc:511] Master 127.5.71.190:40203 requested a full tablet report, sending...
I20260923 20:46:59.379319 12394 ts_manager.cc:194] Registered new tserver with Master: c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811)
I20260923 20:46:59.379487 12198 ts_manager.cc:194] Registered new tserver with Master: c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811)
I20260923 20:46:59.379956 11592 ts_manager.cc:194] Re-registered known tserver with Master: c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811)
I20260923 20:46:59.381044 11592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:42467
I20260923 20:46:59.463513 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11787
I20260923 20:46:59.482579 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:40759
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:40203
--webserver_port=43323
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:46:59.822158 12585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:46:59.822471 12585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:46:59.822567 12585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:46:59.830092 12585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:46:59.830231 12585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:46:59.830348 12585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:46:59.840620 12585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:40759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=43323
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:46:59.842428 12585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:46:59.843912 12585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:46:59.858527 12590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:59.858457 12591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:46:59.858505 12593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:46:59.859651 12585 server_base.cc:1034] running on GCE node
I20260923 20:46:59.860450 12585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:46:59.861423 12585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:46:59.862699 12585 hybrid_clock.cc:648] HybridClock initialized: now 1790196419862630 us; error 58 us; skew 500 ppm
I20260923 20:46:59.863089 12585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:46:59.866401 12585 init.cc:377] Logged in from keytab as oryx/127.5.71.130@KRBTEST.COM (short username oryx)
I20260923 20:46:59.868126 12585 webserver.cc:460] Webserver started at http://127.5.71.130:43323/ using document root <none> and password file <none>
I20260923 20:46:59.868983 12585 fs_manager.cc:359] Metadata directory not provided
I20260923 20:46:59.869158 12585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:46:59.875422 12585 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260923 20:46:59.878472 12600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:59.880133 12585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:46:59.880355 12585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "67123dd3363a4d7db208c7dd58914326"
format_stamp: "Formatted at 2026-09-23 20:46:56 on dist-test-slave-dn29"
server_key: "3aa22b9a3d6a28d0370b0d8489fb32b2"
server_key_iv: "0b4dccb6d03ef98bc4cd9eba252e8a19"
server_key_version: "encryptionkey@0"
I20260923 20:46:59.881543 12585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:46:59.910972 12585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:46:59.915486 12585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:46:59.915778 12585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:46:59.916857 12585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:46:59.918289 12585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:46:59.918409 12585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:59.918550 12585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:46:59.918639 12585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:46:59.953946 12585 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:40759
I20260923 20:46:59.954038 12712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:40759 every 8 connection(s)
I20260923 20:46:59.955560 12585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:46:59.956673 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 12585
I20260923 20:46:59.956974 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11921
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.188@KRBTEST.COM
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.131@KRBTEST.COM
Sep 23 20:46:59 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196419, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
I20260923 20:47:00.000103 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.960250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:41277 (local address 127.5.71.188:35943)
0923 20:46:59.974993 (+ 14743us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.975001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.975030 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:46:59.976461 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.976467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.976566 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.976711 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.980165 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.981444 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.983603 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.984296 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.984429 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.990054 (+ 5625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.990062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.990067 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.990131 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.994420 (+ 4289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.995473 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.995481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.995485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.995658 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:46:59.996318 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:46:59.996323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:46:59.996327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:46:59.996900 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:46:59.997050 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 20:46:59.998939 (+ 1889us) server_negotiation.cc:299] Negotiation successful
0923 20:46:59.999212 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14572,"thread_start_us":14476,"threads_started":1}
I20260923 20:47:00.002874 12722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.959465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.5.71.130:41277)
0923 20:46:59.972545 (+ 13080us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.972606 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.976089 (+ 3483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.977191 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.977200 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.977911 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:59.978597 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.978615 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.981627 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.981632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.983405 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.983421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.984788 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.984797 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.985113 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.986129 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:59.986170 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:59.989665 (+ 3495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:46:59.994592 (+ 4927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.994602 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.994625 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.995316 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.995854 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:46:59.995860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:46:59.995864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:46:59.996166 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:46:59.997154 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:46:59.997166 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:46:59.998454 (+ 1288us) client_negotiation.cc:769] Sending connection context
0923 20:46:59.998831 (+ 377us) client_negotiation.cc:240] Negotiation successful
0923 20:46:59.999657 (+ 826us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12708,"mutex_wait_us":1,"thread_start_us":15619,"threads_started":1}
I20260923 20:47:00.005084 12713 heartbeater.cc:347] Connected to a master server at 127.5.71.188:35943
I20260923 20:47:00.005430 12713 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:00.005653 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.957985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.5.71.131:46615)
0923 20:46:59.973456 (+ 15471us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.973468 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.976482 (+ 3014us) server_negotiation.cc:407] Connection header received
0923 20:46:59.978944 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.978954 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.979253 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.979435 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.981878 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.983522 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.985790 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.986708 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.986849 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.994510 (+ 7661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.994532 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.994558 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.994611 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.998727 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.001551 (+ 2824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.001562 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.001567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.001863 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.002517 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.002522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.002526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.003605 (+ 1079us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.004684 (+ 1079us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.004900 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.005204 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15244,"thread_start_us":15490,"threads_started":1}
I20260923 20:47:00.006223 12713 heartbeater.cc:511] Master 127.5.71.188:35943 requested a full tablet report, sending...
I20260923 20:47:00.007000 12044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:00.008474 12394 ts_manager.cc:194] Registered new tserver with Master: 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:00.008802 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.964005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:50039)
0923 20:46:59.975547 (+ 11542us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.975568 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.976545 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.977198 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.977219 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.978180 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:59.978615 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.978632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.980151 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.980159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.985969 (+ 5810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.985997 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.989836 (+ 3839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.989844 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.990081 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.990704 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:59.990742 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:59.993645 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:00.002066 (+ 8421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.002073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.002076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.002665 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.003438 (+ 773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.003444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.003448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.003676 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.007531 (+ 3855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:00.007536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:00.007742 (+ 206us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.007921 (+ 179us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.008359 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11394,"mutex_wait_us":2988,"thread_start_us":8024,"threads_started":1}
I20260923 20:47:00.008996 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.964019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:50039 (local address 127.5.71.190:40203)
0923 20:46:59.974573 (+ 10554us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.974582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.975678 (+ 1096us) server_negotiation.cc:407] Connection header received
0923 20:46:59.976730 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.976735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.976814 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.976918 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.978792 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.979967 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.987085 (+ 7118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.987835 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.987962 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.993879 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.993888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.993894 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.993943 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:46:59.997702 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.002975 (+ 5273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.002982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.003002 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.003265 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.003904 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.003910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.003913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.004287 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.004457 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.008302 (+ 3845us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.008585 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10401,"thread_start_us":9836,"threads_started":1}
I20260923 20:47:00.009529 12724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.979073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.5.71.130:48003)
0923 20:46:59.987176 (+ 8103us) negotiation.cc:107] Waiting for socket to connect
0923 20:46:59.987204 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:46:59.987336 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:46:59.988827 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:46:59.988833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:46:59.989603 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:46:59.990055 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.990068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.991601 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.991608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.993500 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:46:59.993521 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.994923 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.994934 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:46:59.995164 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.995713 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:46:59.995742 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:46:59.998571 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:00.004238 (+ 5667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.004244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.004248 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.004754 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.005202 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.005207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.005210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.005392 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.007081 (+ 1689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:00.007089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:00.007338 (+ 249us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.007529 (+ 191us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.009107 (+ 1578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7919,"mutex_wait_us":240,"thread_start_us":7725,"threads_started":1}
I20260923 20:47:00.009667 12714 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40203
I20260923 20:47:00.009793 12714 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:00.010272 12714 heartbeater.cc:511] Master 127.5.71.190:40203 requested a full tablet report, sending...
I20260923 20:47:00.010507 12723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:46:59.975967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:48003 (local address 127.5.71.189:35309)
0923 20:46:59.986790 (+ 10823us) server_negotiation.cc:206] Beginning negotiation
0923 20:46:59.986799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:46:59.987450 (+ 651us) server_negotiation.cc:407] Connection header received
0923 20:46:59.987527 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:46:59.987534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:46:59.987634 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:46:59.989519 (+ 1885us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:46:59.990253 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.991403 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.993733 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:46:59.994412 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:46:59.994568 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:46:59.998803 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:46:59.998812 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:46:59.998817 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:46:59.998863 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.004071 (+ 5208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.004902 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.004909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.004912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.005061 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.005528 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.005532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.005535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.006808 (+ 1273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.006991 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.007770 (+ 779us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.010106 (+ 2336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10670,"thread_start_us":10562,"threads_started":1}
I20260923 20:47:00.011425 11592 ts_manager.cc:194] Re-registered known tserver with Master: 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:00.011912 12715 heartbeater.cc:347] Connected to a master server at 127.5.71.189:35309
I20260923 20:47:00.012045 12715 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:00.012466 12715 heartbeater.cc:511] Master 127.5.71.189:35309 requested a full tablet report, sending...
I20260923 20:47:00.012636 11592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:50039
I20260923 20:47:00.016455 12198 ts_manager.cc:194] Registered new tserver with Master: 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:00.099460 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11921
I20260923 20:47:00.117523 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:46615
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:40203
--webserver_port=34115
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:47:00.384080 12574 heartbeater.cc:503] Master 127.5.71.190:40203 was elected leader, sending a full tablet report...
W20260923 20:47:00.456984 12726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:00.457311 12726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:00.457506 12726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:00.464998 12726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:00.465119 12726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:00.465294 12726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:00.476009 12726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:46615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=34115
--tserver_master_addrs=127.5.71.190:40203,127.5.71.189:35309,127.5.71.188:35943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:00.477440 12726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:00.478858 12726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:00.493860 12731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:00.493868 12732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:00.494240 12734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:00.494675 12726 server_base.cc:1034] running on GCE node
I20260923 20:47:00.495362 12726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:00.496464 12726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:00.497684 12726 hybrid_clock.cc:648] HybridClock initialized: now 1790196420497627 us; error 48 us; skew 500 ppm
I20260923 20:47:00.498061 12726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:00 dist-test-slave-dn29 krb5kdc[11547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196420, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:00.501538 12726 init.cc:377] Logged in from keytab as oryx/127.5.71.131@KRBTEST.COM (short username oryx)
I20260923 20:47:00.503247 12726 webserver.cc:460] Webserver started at http://127.5.71.131:34115/ using document root <none> and password file <none>
I20260923 20:47:00.504077 12726 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:00.504233 12726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:00.509835 12726 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 20:47:00.513757 12741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:00.515754 12726 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:47:00.515972 12726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "557016df63df4b268775a1ab2bc4d2cc"
format_stamp: "Formatted at 2026-09-23 20:46:57 on dist-test-slave-dn29"
server_key: "e30a22cf22772be4f2afefa8e13a5f9c"
server_key_iv: "10dd26709bbfd69381bca57e5f86e385"
server_key_version: "encryptionkey@0"
I20260923 20:47:00.517496 12726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:00.536386 12726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:00.540911 12726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:00.541183 12726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:00.542477 12726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:00.544044 12726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:00.544129 12726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:00.544261 12726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:00.544334 12726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:00.579022 12726 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:46615
I20260923 20:47:00.579149 12853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:46615 every 8 connection(s)
I20260923 20:47:00.580858 12726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:00.589571 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 12726
Sep 23 20:47:00 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196420, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.188@KRBTEST.COM
Sep 23 20:47:00 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196420, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
Sep 23 20:47:00 dist-test-slave-dn29 krb5kdc[11547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196420, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.189@KRBTEST.COM
I20260923 20:47:00.620537 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.583862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:36085 (local address 127.5.71.188:35943)
0923 20:47:00.584776 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.584780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.586371 (+ 1591us) server_negotiation.cc:407] Connection header received
0923 20:47:00.588805 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.588812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.588894 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.588994 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:00.593060 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.594539 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.600227 (+ 5688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.600964 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.601042 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.606326 (+ 5284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:00.606333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:00.606339 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:00.606382 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.614312 (+ 7930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.615506 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.615513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.615517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.615691 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.616570 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.616580 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.616586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.617374 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.617549 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.619437 (+ 1888us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.619808 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":712,"threads_started":1}
I20260923 20:47:00.621980 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.584980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.5.71.131:36085)
0923 20:47:00.585893 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.585940 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.588465 (+ 2525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.592046 (+ 3581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.592051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.592591 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:00.592871 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.592882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.597161 (+ 4279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.597165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.599655 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.599672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.601380 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.601387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.601557 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.602117 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:00.602140 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:00.606067 (+ 3927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:00.614522 (+ 8455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.614539 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.614564 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.615326 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.615865 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.615874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.615881 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.616380 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.617694 (+ 1314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:00.617704 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:00.618830 (+ 1126us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.619069 (+ 239us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.619777 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":1637,"thread_start_us":551,"threads_started":1}
I20260923 20:47:00.624519 12854 heartbeater.cc:347] Connected to a master server at 127.5.71.188:35943
I20260923 20:47:00.624778 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.586696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:43217 (local address 127.5.71.189:35309)
0923 20:47:00.587393 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.587399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.587423 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:47:00.588807 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.588812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.588894 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.588994 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:00.591892 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.593385 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.596889 (+ 3504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.597813 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.597939 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.610703 (+ 12764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:00.610713 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:00.610721 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:00.610780 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.615191 (+ 4411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.616312 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.616320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.616326 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.616514 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.616963 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.616970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.616976 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.617395 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.617553 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.623657 (+ 6104us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.624039 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":442,"threads_started":1}
I20260923 20:47:00.625042 12854 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:00.626358 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.586548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.5.71.131:43217)
0923 20:47:00.587216 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.587237 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.588448 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.589187 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.589194 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.590298 (+ 1104us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:00.591585 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.591621 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.593620 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.593629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.596639 (+ 3010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.596660 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.598401 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.598411 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.598934 (+ 523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.601969 (+ 3035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:00.602013 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:00.610433 (+ 8420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:00.615417 (+ 4984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.615424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.615431 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.616135 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.616657 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.616664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.616670 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.616830 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.617694 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:00.617706 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:00.618313 (+ 607us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.625001 (+ 6688us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.625459 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":185,"spinlock_wait_cycles":3840,"thread_start_us":320,"threads_started":1}
I20260923 20:47:00.626772 12854 heartbeater.cc:511] Master 127.5.71.188:35943 requested a full tablet report, sending...
I20260923 20:47:00.629133 12856 heartbeater.cc:347] Connected to a master server at 127.5.71.189:35309
I20260923 20:47:00.629351 12856 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:00.629398 12394 ts_manager.cc:194] Registered new tserver with Master: 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615)
I20260923 20:47:00.630349 12856 heartbeater.cc:511] Master 127.5.71.189:35309 requested a full tablet report, sending...
I20260923 20:47:00.630946 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.583837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:42869 (local address 127.5.71.190:40203)
0923 20:47:00.584306 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.584312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.586073 (+ 1761us) server_negotiation.cc:407] Connection header received
0923 20:47:00.588698 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.588704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.588788 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.588924 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:00.591784 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.592997 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.597128 (+ 4131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.597796 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.597925 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.608711 (+ 10786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:00.608721 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:00.608730 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:00.608787 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.613150 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.618763 (+ 5613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.618769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.618773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.619081 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.621644 (+ 2563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.621658 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.621664 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.622291 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.622436 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.629946 (+ 7510us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.630345 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":230,"threads_started":1}
I20260923 20:47:00.632274 12198 ts_manager.cc:194] Registered new tserver with Master: 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615)
I20260923 20:47:00.635691 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.584686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:42869)
0923 20:47:00.585892 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.585940 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.588426 (+ 2486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.589185 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.589193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.590313 (+ 1120us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:00.591585 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.591621 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.594790 (+ 3169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.594798 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.596980 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.597000 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.598781 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.598790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.599860 (+ 1070us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.602425 (+ 2565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:00.602450 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:00.608207 (+ 5757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:00.617879 (+ 9672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.617884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.617890 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.618585 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.621027 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:00.621034 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:00.621040 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:00.621457 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:00.622589 (+ 1132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:00.622595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:00.628513 (+ 5918us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.628677 (+ 164us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.629093 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":873,"spinlock_wait_cycles":4992,"thread_start_us":618,"threads_started":1}
I20260923 20:47:00.638121 12855 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40203
I20260923 20:47:00.638294 12855 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:00.638752 12855 heartbeater.cc:511] Master 127.5.71.190:40203 requested a full tablet report, sending...
I20260923 20:47:00.639871 11592 ts_manager.cc:194] Re-registered known tserver with Master: 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615)
I20260923 20:47:00.641081 11592 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:42869
I20260923 20:47:00.645431 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.603079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.5.71.189:35309)
0923 20:47:00.606342 (+ 3263us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.606348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.609224 (+ 2876us) server_negotiation.cc:407] Connection header received
0923 20:47:00.609297 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.609302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.609416 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.609528 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:00.612427 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.613866 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.617846 (+ 3980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.618629 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.618748 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.637948 (+ 19200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:00.637957 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:00.637963 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:00.638027 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.641747 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.642772 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.642778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.642782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.642949 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.643375 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.643380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.643385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.643808 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.644062 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.644751 (+ 689us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.644985 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1923,"thread_start_us":1804,"threads_started":1}
I20260923 20:47:00.651587 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.607639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40132 (local address 127.5.71.188:35943)
0923 20:47:00.620917 (+ 13278us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.620925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.626164 (+ 5239us) server_negotiation.cc:407] Connection header received
0923 20:47:00.635359 (+ 9195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.635369 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.635476 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.635584 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:00.638700 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.640223 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.642065 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.642702 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.642918 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.646141 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:00.646147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:00.646152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:00.646195 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.648836 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.649599 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.649604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.649609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.649740 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.650112 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.650123 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.650125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.650382 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.650665 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.651000 (+ 335us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.651221 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13165,"thread_start_us":20402,"threads_started":1}
I20260923 20:47:00.654147 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.603036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.5.71.190:40203)
0923 20:47:00.604704 (+ 1668us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.604713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.625863 (+ 21150us) server_negotiation.cc:407] Connection header received
0923 20:47:00.631150 (+ 5287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.631155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.631242 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.631329 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:00.635075 (+ 3746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.636776 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.641442 (+ 4666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.644025 (+ 2583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.644215 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.648511 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:00.648516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:00.648520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:00.648557 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:00.651150 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.652058 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.652062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.652064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.652191 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:00.652620 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:00.652625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:00.652628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:00.653008 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:00.653258 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.653634 (+ 376us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.653871 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1496,"thread_start_us":2097,"threads_started":1}
I20260923 20:47:00.658566 11592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:00.665876 11592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:00.697903 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.684951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46615 (local address 127.0.0.1:51684)
0923 20:47:00.688843 (+ 3892us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.688860 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.688971 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.690548 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.690556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.690593 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:00.691090 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.691106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.693077 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.693084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.694510 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.694522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.697087 (+ 2565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.697094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.697256 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.697290 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.697384 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.697537 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3782,"thread_start_us":3029,"threads_started":1}
I20260923 20:47:00.697993 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.685202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.5.71.131:46615)
0923 20:47:00.685903 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.685914 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.689963 (+ 4049us) server_negotiation.cc:407] Connection header received
0923 20:47:00.690069 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.690078 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.690354 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.690547 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:00.691378 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.692965 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.694672 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.696879 (+ 2207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.697018 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.697254 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.697455 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.697641 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":323,"threads_started":1}
I20260923 20:47:00.698438 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.684332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.5.71.129:36811)
0923 20:47:00.684857 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.684867 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.688331 (+ 3464us) server_negotiation.cc:407] Connection header received
0923 20:47:00.688560 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.688572 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.688893 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.689100 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:00.689936 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.691327 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.695610 (+ 4283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.697059 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.697164 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.697338 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.698026 (+ 688us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.698202 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":261,"threads_started":1}
I20260923 20:47:00.699314 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.689141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.5.71.130:40759)
0923 20:47:00.689695 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.689705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.692446 (+ 2741us) server_negotiation.cc:407] Connection header received
0923 20:47:00.692712 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.692719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.692959 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.693131 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:00.693870 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.694789 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.696813 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.698169 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.698257 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.698395 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.698888 (+ 493us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.699047 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":254,"threads_started":1}
I20260923 20:47:00.699338 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.688900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40759 (local address 127.0.0.1:53576)
0923 20:47:00.692334 (+ 3434us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.692352 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.692546 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.693334 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.693340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.693359 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:00.693714 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.693721 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.694925 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.694933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.696642 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.696655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.698576 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.698584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.698724 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.698746 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.698846 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.698974 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3271,"thread_start_us":3107,"threads_started":1}
I20260923 20:47:00.699781 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.684051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:36811 (local address 127.0.0.1:37064)
0923 20:47:00.688199 (+ 4148us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.688214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.688372 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.689296 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.689303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.689330 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:00.689738 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.689788 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.691487 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.691495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.695415 (+ 3920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.695429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.697563 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.697570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.697780 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.697814 (+ 34us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.699031 (+ 1217us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.699182 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4045,"thread_start_us":4563,"threads_started":1}
I20260923 20:47:00.702857 12789 tablet_service.cc:1467] Processing CreateTablet for tablet 14781dec740249dba8da79762895b117 (DEFAULT_TABLE table=test-table [id=0fb560379397455f89ca5fc45edf38ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:00.703110 12648 tablet_service.cc:1467] Processing CreateTablet for tablet 14781dec740249dba8da79762895b117 (DEFAULT_TABLE table=test-table [id=0fb560379397455f89ca5fc45edf38ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:00.703110 12508 tablet_service.cc:1467] Processing CreateTablet for tablet 14781dec740249dba8da79762895b117 (DEFAULT_TABLE table=test-table [id=0fb560379397455f89ca5fc45edf38ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:00.704306 12789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14781dec740249dba8da79762895b117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:00.704440 12508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14781dec740249dba8da79762895b117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:00.704445 12648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14781dec740249dba8da79762895b117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:00.717751 12892 tablet_bootstrap.cc:492] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Bootstrap starting.
I20260923 20:47:00.722065 12892 tablet_bootstrap.cc:654] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:00.723213 12892 log.cc:824] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:00.726368 12892 tablet_bootstrap.cc:492] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: No bootstrap required, opened a new log
I20260923 20:47:00.726723 12892 ts_tablet_manager.cc:1397] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260923 20:47:00.728606 12894 tablet_bootstrap.cc:492] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Bootstrap starting.
I20260923 20:47:00.728850 12893 tablet_bootstrap.cc:492] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Bootstrap starting.
I20260923 20:47:00.731904 12893 tablet_bootstrap.cc:654] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:00.733335 12893 log.cc:824] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:00.734499 12892 raft_consensus.cc:348] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.734987 12892 raft_consensus.cc:374] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:00.735090 12892 raft_consensus.cc:729] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 557016df63df4b268775a1ab2bc4d2cc, State: Initialized, Role: FOLLOWER
I20260923 20:47:00.735324 12894 tablet_bootstrap.cc:654] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:00.735894 12892 consensus_queue.cc:260] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.736617 12894 log.cc:824] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:00.736891 12893 tablet_bootstrap.cc:492] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: No bootstrap required, opened a new log
I20260923 20:47:00.737196 12893 ts_tablet_manager.cc:1397] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260923 20:47:00.740384 12855 heartbeater.cc:503] Master 127.5.71.190:40203 was elected leader, sending a full tablet report...
I20260923 20:47:00.741122 12892 ts_tablet_manager.cc:1428] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260923 20:47:00.745366 12894 tablet_bootstrap.cc:492] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: No bootstrap required, opened a new log
I20260923 20:47:00.745747 12894 ts_tablet_manager.cc:1397] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.010s
I20260923 20:47:00.750334 12893 raft_consensus.cc:348] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.750734 12893 raft_consensus.cc:374] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:00.750890 12893 raft_consensus.cc:729] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67123dd3363a4d7db208c7dd58914326, State: Initialized, Role: FOLLOWER
I20260923 20:47:00.751552 12893 consensus_queue.cc:260] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.752707 12714 heartbeater.cc:503] Master 127.5.71.190:40203 was elected leader, sending a full tablet report...
I20260923 20:47:00.753414 12893 ts_tablet_manager.cc:1428] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260923 20:47:00.754498 12894 raft_consensus.cc:348] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.754992 12894 raft_consensus.cc:374] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:00.755090 12894 raft_consensus.cc:729] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b4262b58f540f2bb5432a28e9e2c3b, State: Initialized, Role: FOLLOWER
I20260923 20:47:00.755858 12894 consensus_queue.cc:260] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.758590 12894 ts_tablet_manager.cc:1428] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260923 20:47:00.834923 12857 tablet.cc:2369] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:47:00.834923 12576 tablet.cc:2369] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:47:00.928143 12897 raft_consensus.cc:482] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:00.928491 12897 raft_consensus.cc:504] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.930282 12897 leader_election.cc:288] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811), 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:00.936513 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.930657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:57337 (local address 127.5.71.129:36811)
0923 20:47:00.930876 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.930885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.930907 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:47:00.931121 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.931126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.931277 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.931383 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:00.931862 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.932658 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.934684 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.935596 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.935776 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.935878 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.936162 (+ 284us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.936276 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:47:00.936513 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.931192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:35903 (local address 127.5.71.130:40759)
0923 20:47:00.931422 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:00.931427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:00.931446 (+ 19us) server_negotiation.cc:407] Connection header received
0923 20:47:00.931510 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:00.931514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:00.931580 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:00.931680 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:00.932088 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.932787 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.934587 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.935550 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.935663 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.935764 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:00.936120 (+ 356us) server_negotiation.cc:299] Negotiation successful
0923 20:47:00.936242 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 20:47:00.936530 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.930927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40759 (local address 127.5.71.131:35903)
0923 20:47:00.931208 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.931230 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.931356 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.931704 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.931707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.931727 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:00.931948 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.931959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.932917 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.932920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.934413 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.934427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.935713 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.935719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.935948 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.935985 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.936059 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.936212 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":5}
I20260923 20:47:00.936545 12860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:00.930499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:36811 (local address 127.5.71.131:57337)
0923 20:47:00.930692 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:00.930737 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:00.930909 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:00.931436 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:00.931442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:00.931472 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:00.931666 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.931677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.932829 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.932837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.934517 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:00.934533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:00.935825 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:00.935831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:00.935989 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:00.936011 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 20:47:00.936123 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 20:47:00.936247 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":7}
I20260923 20:47:00.937594 12668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14781dec740249dba8da79762895b117" candidate_uuid: "557016df63df4b268775a1ab2bc4d2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67123dd3363a4d7db208c7dd58914326" is_pre_election: true
I20260923 20:47:00.937700 12528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14781dec740249dba8da79762895b117" candidate_uuid: "557016df63df4b268775a1ab2bc4d2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" is_pre_election: true
I20260923 20:47:00.938151 12668 raft_consensus.cc:2454] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 557016df63df4b268775a1ab2bc4d2cc in term 0.
I20260923 20:47:00.938194 12528 raft_consensus.cc:2454] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 557016df63df4b268775a1ab2bc4d2cc in term 0.
I20260923 20:47:00.938836 12745 leader_election.cc:302] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 557016df63df4b268775a1ab2bc4d2cc, 67123dd3363a4d7db208c7dd58914326; no voters:
I20260923 20:47:00.939209 12897 raft_consensus.cc:2781] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:00.939368 12897 raft_consensus.cc:482] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:00.939481 12897 raft_consensus.cc:3037] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:00.941996 12897 raft_consensus.cc:504] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.942698 12897 leader_election.cc:288] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [CANDIDATE]: Term 1 election: Requested vote from peers c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811), 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:00.943269 12528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14781dec740249dba8da79762895b117" candidate_uuid: "557016df63df4b268775a1ab2bc4d2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b"
I20260923 20:47:00.943498 12528 raft_consensus.cc:3037] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:00.943629 12668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14781dec740249dba8da79762895b117" candidate_uuid: "557016df63df4b268775a1ab2bc4d2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67123dd3363a4d7db208c7dd58914326"
I20260923 20:47:00.943871 12668 raft_consensus.cc:3037] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:00.946727 12528 raft_consensus.cc:2454] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 557016df63df4b268775a1ab2bc4d2cc in term 1.
I20260923 20:47:00.947008 12668 raft_consensus.cc:2454] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 557016df63df4b268775a1ab2bc4d2cc in term 1.
I20260923 20:47:00.947347 12745 leader_election.cc:302] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 557016df63df4b268775a1ab2bc4d2cc, c8b4262b58f540f2bb5432a28e9e2c3b; no voters:
I20260923 20:47:00.947702 12897 raft_consensus.cc:2781] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:00.948230 12897 raft_consensus.cc:686] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 1 LEADER]: Becoming Leader. State: Replica: 557016df63df4b268775a1ab2bc4d2cc, State: Running, Role: LEADER
I20260923 20:47:00.949011 12897 consensus_queue.cc:237] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } }
I20260923 20:47:00.954345 11592 catalog_manager.cc:5495] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131). New cstate: current_term: 1 leader_uuid: "557016df63df4b268775a1ab2bc4d2cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } health_report { overall_health: HEALTHY } } }
W20260923 20:47:00.971629 12720 tablet.cc:2369] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:47:01.005748 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:01.000077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51698 (local address 127.5.71.131:46615)
0923 20:47:01.000354 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:01.000359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:01.000380 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:47:01.000598 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:01.000601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:01.000681 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:01.000815 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:01.001203 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.001938 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.003403 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.003896 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.003990 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:01.004508 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:01.005070 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:01.005190 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:01.005359 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 20:47:01.005431 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260923 20:47:01.015194 12668 raft_consensus.cc:1261] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Refusing update from remote peer 557016df63df4b268775a1ab2bc4d2cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:01.015194 12528 raft_consensus.cc:1261] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Refusing update from remote peer 557016df63df4b268775a1ab2bc4d2cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:01.016531 12901 consensus_queue.cc:1035] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:01.017015 12897 consensus_queue.cc:1035] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:01.033002 12905 mvcc.cc:205] Tried to move back new op lower bound from 7332644540465983488 to 7332644540210999296. Current Snapshot: MvccSnapshot[applied={T|T < 7332644540465983488}]
I20260923 20:47:01.049271 11591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:01.049600 11591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:01.056452 11591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac3d81454f8c428d9d0567c0acb9b38e: Sending DeleteTablet for 3 replicas of tablet 14781dec740249dba8da79762895b117
I20260923 20:47:01.057787 12508 tablet_service.cc:1514] Processing DeleteTablet for tablet 14781dec740249dba8da79762895b117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:01 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:37064
I20260923 20:47:01.058061 12648 tablet_service.cc:1514] Processing DeleteTablet for tablet 14781dec740249dba8da79762895b117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:01 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:53576
I20260923 20:47:01.058281 12789 tablet_service.cc:1514] Processing DeleteTablet for tablet 14781dec740249dba8da79762895b117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:01 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:51684
I20260923 20:47:01.070621 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 11560
I20260923 20:47:01.071440 12916 tablet_replica.cc:330] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: stopping tablet replica
I20260923 20:47:01.071941 11622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:01.072405 12915 tablet_replica.cc:330] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: stopping tablet replica
I20260923 20:47:01.072888 12915 raft_consensus.cc:2229] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:01.073189 12916 raft_consensus.cc:2229] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:01.073451 12915 raft_consensus.cc:2258] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:01.074378 12916 raft_consensus.cc:2258] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:01.074895 12914 tablet_replica.cc:330] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: stopping tablet replica
I20260923 20:47:01.075416 12914 raft_consensus.cc:2229] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:01.076092 12914 raft_consensus.cc:2258] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:01.078852 12916 ts_tablet_manager.cc:1905] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:01.079387 12914 ts_tablet_manager.cc:1905] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:01.087435 12914 ts_tablet_manager.cc:1918] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:01.087749 12914 log.cc:1196] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/14781dec740249dba8da79762895b117
I20260923 20:47:01.088967 12914 ts_tablet_manager.cc:1939] T 14781dec740249dba8da79762895b117 P c8b4262b58f540f2bb5432a28e9e2c3b: Deleting consensus metadata
I20260923 20:47:01.089548 12915 ts_tablet_manager.cc:1905] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:01.094205 12916 ts_tablet_manager.cc:1918] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:01.094410 12916 log.cc:1196] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/14781dec740249dba8da79762895b117
I20260923 20:47:01.095038 12916 ts_tablet_manager.cc:1939] T 14781dec740249dba8da79762895b117 P 557016df63df4b268775a1ab2bc4d2cc: Deleting consensus metadata
I20260923 20:47:01.098521 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:01.061741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58494 (local address 127.5.71.189:35309)
0923 20:47:01.061932 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:01.061938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:01.068754 (+ 6816us) server_negotiation.cc:407] Connection header received
0923 20:47:01.068990 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:01.068997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:01.069099 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:01.069205 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:01.075957 (+ 6752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.077079 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.082196 (+ 5117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.082895 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.083002 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:01.090491 (+ 7489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:01.090499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:01.090505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:01.090558 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:01.094388 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.095343 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.095349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.095353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.095505 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.096019 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.096024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.096028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.096409 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:01.096654 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:01.097865 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0923 20:47:01.098132 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:47:01.098899 12915 ts_tablet_manager.cc:1918] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:01.099073 12915 log.cc:1196] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/14781dec740249dba8da79762895b117
I20260923 20:47:01.099630 12915 ts_tablet_manager.cc:1939] T 14781dec740249dba8da79762895b117 P 67123dd3363a4d7db208c7dd58914326: Deleting consensus metadata
I20260923 20:47:01.117846 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:01.068980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.5.71.188:35943)
0923 20:47:01.069486 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:01.069494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:01.100674 (+ 31180us) server_negotiation.cc:407] Connection header received
0923 20:47:01.100933 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:01.100939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:01.101024 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:01.101095 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:01.103139 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.104432 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.106434 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.107155 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.107318 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:01.110832 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:01.110838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:01.110842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:01.110898 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:01.114713 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.115582 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.115588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.115591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.115748 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.116226 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.116231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.116235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.116616 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:01.116867 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:01.117242 (+ 375us) server_negotiation.cc:299] Negotiation successful
0923 20:47:01.117477 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373}
I20260923 20:47:01.230504 11577 catalog_manager.cc:4841] TS 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615): tablet 14781dec740249dba8da79762895b117 (table test-table [id=0fb560379397455f89ca5fc45edf38ad]) successfully deleted
I20260923 20:47:01.230883 11579 catalog_manager.cc:4841] TS 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759): tablet 14781dec740249dba8da79762895b117 (table test-table [id=0fb560379397455f89ca5fc45edf38ad]) successfully deleted
I20260923 20:47:01.231467 11579 catalog_manager.cc:4841] TS c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811): tablet 14781dec740249dba8da79762895b117 (table test-table [id=0fb560379397455f89ca5fc45edf38ad]) successfully deleted
I20260923 20:47:01.231612 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 11560
I20260923 20:47:01.278090 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:01.259663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58500 (local address 127.5.71.189:35309)
0923 20:47:01.260070 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:01.260077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:01.260649 (+ 572us) server_negotiation.cc:407] Connection header received
0923 20:47:01.260937 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:01.260943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:01.261027 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:01.261099 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:01.263393 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.264539 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.266617 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.267215 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.267316 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:01.271444 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:01.271451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:01.271456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:01.271499 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:01.275047 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.275884 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.275891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.275895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.276044 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.276471 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.276482 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.276485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.276778 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:01.277042 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:01.277447 (+ 405us) server_negotiation.cc:299] Negotiation successful
0923 20:47:01.277682 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260923 20:47:01.284718 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:01.261094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.5.71.188:35943)
0923 20:47:01.261358 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:01.261367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:01.267211 (+ 5844us) server_negotiation.cc:407] Connection header received
0923 20:47:01.268057 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:01.268066 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:01.268181 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:01.268268 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:01.271720 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.272875 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.274771 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:01.275432 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:01.275566 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:01.279126 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:01.279131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:01.279134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:01.279200 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:01.281880 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.282644 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.282648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.282650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.282787 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:01.283164 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:01.283169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:01.283172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:01.283473 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:01.283737 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:01.284111 (+ 374us) server_negotiation.cc:299] Negotiation successful
0923 20:47:01.284404 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 20:47:02.042301 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.041238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:56653)
0923 20:47:02.041978 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.042136 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":547,"threads_started":1}
W20260923 20:47:02.042984 12574 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:40203 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.044624 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.043559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:42187)
0923 20:47:02.044333 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.044455 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":581,"threads_started":1}
W20260923 20:47:02.045311 12855 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:40203 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.045469 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.044278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:42157)
0923 20:47:02.045012 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.045242 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":254,"threads_started":1}
W20260923 20:47:02.045828 12714 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:40203 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.557313 12929 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ac3d81454f8c428d9d0567c0acb9b38e)
I20260923 20:47:02.557582 12929 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } }
I20260923 20:47:02.557675 12930 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ac3d81454f8c428d9d0567c0acb9b38e)
I20260923 20:47:02.558038 12930 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } }
I20260923 20:47:02.558504 12929 leader_election.cc:288] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203), dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189:35309)
I20260923 20:47:02.566049 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.558697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43726)
0923 20:47:02.565454 (+ 6757us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.565685 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6578,"thread_start_us":6950,"threads_started":1}
I20260923 20:47:02.566167 12930 leader_election.cc:288] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203), d0832bd8d4464f52b4c15575cd4a6d6a (127.5.71.188:35943)
W20260923 20:47:02.566744 12381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.574478 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.573997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43754)
0923 20:47:02.574215 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.574305 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260923 20:47:02.575919 12381 leader_election.cc:334] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203): Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.576546 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.558896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58506 (local address 127.5.71.189:35309)
0923 20:47:02.565526 (+ 6630us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.565532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.566421 (+ 889us) server_negotiation.cc:407] Connection header received
0923 20:47:02.566715 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.566749 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.566841 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.566922 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.567990 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.569526 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.573149 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.574492 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.574641 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.574737 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.575780 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.576022 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6510,"thread_start_us":6609,"threads_started":1}
I20260923 20:47:02.577281 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.558767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:35309 (local address 127.0.0.1:58506)
0923 20:47:02.566291 (+ 7524us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.566311 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.566534 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.567190 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.567197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.567236 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.567710 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.567728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.569848 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.569857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.572915 (+ 3058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.572941 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.575168 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.575177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.575483 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.575541 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.576734 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.576930 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7386,"thread_start_us":7243,"threads_started":1}
I20260923 20:47:02.578298 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.560068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43734)
0923 20:47:02.577323 (+ 17255us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.577878 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17072,"thread_start_us":17500,"threads_started":1}
W20260923 20:47:02.578912 12185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.579447 12208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" is_pre_election: true
I20260923 20:47:02.579748 12208 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0832bd8d4464f52b4c15575cd4a6d6a in term 1.
I20260923 20:47:02.580813 12378 leader_election.cc:302] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d0832bd8d4464f52b4c15575cd4a6d6a, dfe3239d3f6d4b65816c1217a51df7d4; no voters: ac3d81454f8c428d9d0567c0acb9b38e
I20260923 20:47:02.581455 12929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 20:47:02.581566 12929 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Starting leader election (detected failure of leader ac3d81454f8c428d9d0567c0acb9b38e)
I20260923 20:47:02.581642 12929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 1 FOLLOWER]: Advancing to term 2
I20260923 20:47:02.584481 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.584025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43762)
0923 20:47:02.584221 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.584331 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260923 20:47:02.585379 12185 leader_election.cc:334] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203): Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.585369 12929 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } }
I20260923 20:47:02.586174 12929 leader_election.cc:288] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 election: Requested vote from peers ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203), dfe3239d3f6d4b65816c1217a51df7d4 (127.5.71.189:35309)
I20260923 20:47:02.586704 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.586243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43774)
0923 20:47:02.586441 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.586553 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260923 20:47:02.586694 12208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dfe3239d3f6d4b65816c1217a51df7d4"
I20260923 20:47:02.586951 12208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260923 20:47:02.587242 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.566950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.5.71.188:35943)
0923 20:47:02.573013 (+ 6063us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.573021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.578034 (+ 5013us) server_negotiation.cc:407] Connection header received
0923 20:47:02.578217 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.578225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.578317 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.578431 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.579162 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.581055 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.584452 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.585754 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.585914 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.586042 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.586592 (+ 550us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.586740 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5863,"thread_start_us":5689,"threads_started":1}
I20260923 20:47:02.587399 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.574939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:35943 (local address 127.0.0.1:40174)
0923 20:47:02.577910 (+ 2971us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.577929 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.578068 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.578523 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.578529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.578564 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.578918 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.578930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.581304 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.581314 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.584279 (+ 2965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.584293 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.585948 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.585955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.586159 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.586198 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.586442 (+ 244us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.586647 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2844,"thread_start_us":1994,"threads_started":1}
I20260923 20:47:02.588145 12404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" is_pre_election: true
I20260923 20:47:02.588614 12404 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dfe3239d3f6d4b65816c1217a51df7d4 in current term 2: Already voted for candidate d0832bd8d4464f52b4c15575cd4a6d6a in this term.
I20260923 20:47:02.589107 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.588665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43784)
0923 20:47:02.588863 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.588958 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260923 20:47:02.589298 12381 leader_election.cc:334] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203): Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
I20260923 20:47:02.589327 12182 leader_election.cc:302] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dfe3239d3f6d4b65816c1217a51df7d4; no voters: ac3d81454f8c428d9d0567c0acb9b38e, d0832bd8d4464f52b4c15575cd4a6d6a
I20260923 20:47:02.590029 12208 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0832bd8d4464f52b4c15575cd4a6d6a in term 2.
I20260923 20:47:02.590296 12930 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:47:02.590476 12378 leader_election.cc:302] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d0832bd8d4464f52b4c15575cd4a6d6a, dfe3239d3f6d4b65816c1217a51df7d4; no voters: ac3d81454f8c428d9d0567c0acb9b38e
I20260923 20:47:02.590773 12929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 2 FOLLOWER]: Leader election won for term 2
I20260923 20:47:02.591329 12929 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [term 2 LEADER]: Becoming Leader. State: Replica: d0832bd8d4464f52b4c15575cd4a6d6a, State: Running, Role: LEADER
I20260923 20:47:02.591842 12929 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } }
I20260923 20:47:02.593600 12939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0832bd8d4464f52b4c15575cd4a6d6a. Latest consensus state: current_term: 2 leader_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:47:02.594002 12939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:02.594761 12941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:02.598361 12941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:02.599186 12941 catalog_manager.cc:1259] Loaded cluster ID: 667c948b58ec4820b40940210c9013a2
I20260923 20:47:02.599289 12941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:02.600471 12941 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:02.601305 12941 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Loaded TSK: 0
I20260923 20:47:02.602161 12941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:47:02.637058 12393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:02.644874 12393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:02.649307 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.648877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43800)
0923 20:47:02.649053 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.649132 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 20:47:02.649335 12208 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [term 2 FOLLOWER]: Refusing update from remote peer d0832bd8d4464f52b4c15575cd4a6d6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260923 20:47:02.650282 12939 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 20:47:02.650497 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.650057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43812)
0923 20:47:02.650268 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.650353 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260923 20:47:02.650669 12381 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a -> Peer ac3d81454f8c428d9d0567c0acb9b38e (127.5.71.190:40203): Couldn't send request to peer ac3d81454f8c428d9d0567c0acb9b38e. Status: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 20:47:02.653249 12930 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0832bd8d4464f52b4c15575cd4a6d6a. Latest consensus state: current_term: 2 leader_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:47:02.653532 12930 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:47:02.656018 12930 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:47:02.656306 12930 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfe3239d3f6d4b65816c1217a51df7d4 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:47:02.664584 12929 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ac3d81454f8c428d9d0567c0acb9b38e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40203 } } peers { permanent_uuid: "dfe3239d3f6d4b65816c1217a51df7d4" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 35309 } attrs { promote: false } } peers { permanent_uuid: "d0832bd8d4464f52b4c15575cd4a6d6a" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 35943 } attrs { promote: false } } }
I20260923 20:47:02.665028 12929 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:02.682560 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.673451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.5.71.131:46615)
0923 20:47:02.674129 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.674140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.674167 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:47:02.674263 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.674272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.674397 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.675362 (+ 965us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.675602 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.677264 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.680286 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.681490 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.681648 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.681771 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.682100 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.682244 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":335,"threads_started":1}
I20260923 20:47:02.682574 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.673100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46615 (local address 127.0.0.1:51710)
0923 20:47:02.673454 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.673472 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.673611 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.674624 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.674633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.674703 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.675162 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.675180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.677469 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.677478 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.680094 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.680110 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.681715 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.681722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.681912 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.681945 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.682060 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.682219 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"spinlock_wait_cycles":1920}
I20260923 20:47:02.683547 12789 tablet_service.cc:1467] Processing CreateTablet for tablet 8c2192b9608344c08a128f99e55910ab (DEFAULT_TABLE table=test-table [id=560f8d3e7e5a404f890f9abeb3eac6bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:02.684092 12789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c2192b9608344c08a128f99e55910ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:02.688452 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.672710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:36811 (local address 127.0.0.1:37078)
0923 20:47:02.672900 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.672923 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.673033 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.681697 (+ 8664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.681704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.681733 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.681983 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.681994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.683645 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.683650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.685782 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.685795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.687530 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.687536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.687770 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.687851 (+ 81us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.687964 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.688122 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":21}
I20260923 20:47:02.689378 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.674378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.5.71.130:40759)
0923 20:47:02.680348 (+ 5970us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.680358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.680386 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:47:02.680484 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.680493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.680616 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.680762 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.681427 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.682629 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.686730 (+ 4101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.688066 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.688198 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.688305 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.688900 (+ 595us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.689053 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5784,"thread_start_us":5647,"threads_started":1}
I20260923 20:47:02.689633 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.672840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.5.71.129:36811)
0923 20:47:02.681292 (+ 8452us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.681297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.681312 (+ 15us) server_negotiation.cc:407] Connection header received
0923 20:47:02.681418 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.681423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.681517 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.682135 (+ 618us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.682292 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.683486 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.685943 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.687331 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.687484 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.687592 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.689130 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.689278 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8341,"thread_start_us":8224,"threads_started":1}
I20260923 20:47:02.690392 12508 tablet_service.cc:1467] Processing CreateTablet for tablet 8c2192b9608344c08a128f99e55910ab (DEFAULT_TABLE table=test-table [id=560f8d3e7e5a404f890f9abeb3eac6bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:02.690519 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.673968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40759 (local address 127.0.0.1:53586)
0923 20:47:02.674555 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.674575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.674702 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.680792 (+ 6090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.680799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.680829 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.681216 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.681228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.683159 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.683165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.686535 (+ 3370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.686549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.688507 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.688514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.688680 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.688728 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.689575 (+ 847us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.690082 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":64,"spinlock_wait_cycles":2816,"thread_start_us":543,"threads_started":1}
I20260923 20:47:02.690907 12508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c2192b9608344c08a128f99e55910ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:02.691495 12648 tablet_service.cc:1467] Processing CreateTablet for tablet 8c2192b9608344c08a128f99e55910ab (DEFAULT_TABLE table=test-table [id=560f8d3e7e5a404f890f9abeb3eac6bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:02.691972 12648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c2192b9608344c08a128f99e55910ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:02.698611 12953 tablet_bootstrap.cc:492] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Bootstrap starting.
I20260923 20:47:02.699412 12954 tablet_bootstrap.cc:492] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Bootstrap starting.
I20260923 20:47:02.699900 12952 tablet_bootstrap.cc:492] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Bootstrap starting.
I20260923 20:47:02.701151 12954 tablet_bootstrap.cc:654] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:02.701671 12953 tablet_bootstrap.cc:654] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:02.701864 12952 tablet_bootstrap.cc:654] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:02.703840 12954 tablet_bootstrap.cc:492] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: No bootstrap required, opened a new log
I20260923 20:47:02.704003 12954 ts_tablet_manager.cc:1397] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260923 20:47:02.704674 12952 tablet_bootstrap.cc:492] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: No bootstrap required, opened a new log
I20260923 20:47:02.704828 12952 ts_tablet_manager.cc:1397] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 20:47:02.704879 12954 raft_consensus.cc:348] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.705148 12954 raft_consensus.cc:374] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:02.705268 12954 raft_consensus.cc:729] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67123dd3363a4d7db208c7dd58914326, State: Initialized, Role: FOLLOWER
I20260923 20:47:02.705528 12954 consensus_queue.cc:260] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.705698 12952 raft_consensus.cc:348] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.705953 12952 raft_consensus.cc:374] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:02.706096 12952 raft_consensus.cc:729] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 557016df63df4b268775a1ab2bc4d2cc, State: Initialized, Role: FOLLOWER
I20260923 20:47:02.706390 12952 consensus_queue.cc:260] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.708364 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.707392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:41003)
0923 20:47:02.707915 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.708071 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"threads_started":1}
I20260923 20:47:02.708577 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.707628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:50623)
0923 20:47:02.708162 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.708290 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"threads_started":1}
I20260923 20:47:02.710685 12952 ts_tablet_manager.cc:1428] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260923 20:47:02.710925 12854 heartbeater.cc:503] Master 127.5.71.188:35943 was elected leader, sending a full tablet report...
I20260923 20:47:02.711313 12954 ts_tablet_manager.cc:1428] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260923 20:47:02.711544 12713 heartbeater.cc:503] Master 127.5.71.188:35943 was elected leader, sending a full tablet report...
I20260923 20:47:02.713737 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.712840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:36467)
0923 20:47:02.713478 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.713576 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527}
I20260923 20:47:02.714212 12953 tablet_bootstrap.cc:492] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: No bootstrap required, opened a new log
I20260923 20:47:02.714376 12953 ts_tablet_manager.cc:1397] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260923 20:47:02.714689 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.713928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:60785)
0923 20:47:02.714218 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.714537 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260923 20:47:02.715124 12953 raft_consensus.cc:348] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.715343 12953 raft_consensus.cc:374] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:02.715451 12953 raft_consensus.cc:729] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b4262b58f540f2bb5432a28e9e2c3b, State: Initialized, Role: FOLLOWER
I20260923 20:47:02.715699 12953 consensus_queue.cc:260] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.716640 12953 ts_tablet_manager.cc:1428] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 20:47:02.718199 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.717024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:33193)
0923 20:47:02.717679 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.717863 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":371,"threads_started":1}
I20260923 20:47:02.718597 12573 heartbeater.cc:503] Master 127.5.71.188:35943 was elected leader, sending a full tablet report...
I20260923 20:47:02.719233 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.718850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:55869)
0923 20:47:02.719008 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.719099 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 20:47:02.852823 12962 raft_consensus.cc:482] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:02.853101 12962 raft_consensus.cc:504] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.854732 12962 leader_election.cc:288] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615), 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:02.863332 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.855475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:35647 (local address 127.5.71.130:40759)
0923 20:47:02.855757 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.855765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.855812 (+ 47us) server_negotiation.cc:407] Connection header received
0923 20:47:02.855900 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.855908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.856000 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.856083 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.856707 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.857881 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.860793 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.862009 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.862168 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.862252 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.862907 (+ 655us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.863081 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:47:02.863344 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.855138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:45173 (local address 127.5.71.131:46615)
0923 20:47:02.855369 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.855378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.856150 (+ 772us) server_negotiation.cc:407] Connection header received
0923 20:47:02.856355 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.856361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.856434 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.856567 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:02.857113 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.858201 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.860696 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.861987 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.862190 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.862255 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.862907 (+ 652us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.863061 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 20:47:02.863466 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.855192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:46615 (local address 127.5.71.129:45173)
0923 20:47:02.856014 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.856029 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.856180 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.856584 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.856591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.856620 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.856945 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.856957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.858354 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.858359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.860511 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.860520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.862245 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.862251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.862557 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.862744 (+ 187us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.862847 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.863035 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":214,"spinlock_wait_cycles":384,"thread_start_us":503,"threads_started":1}
I20260923 20:47:02.863977 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.855024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40759 (local address 127.5.71.129:35647)
0923 20:47:02.855280 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.855306 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:02.855480 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:02.856133 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:02.856138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:02.856180 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:02.856517 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.856532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.858028 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.858032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.860491 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:02.860512 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.862250 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.862254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:02.862550 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.862745 (+ 195us) client_negotiation.cc:769] Sending connection context
0923 20:47:02.862847 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 20:47:02.863035 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":129,"spinlock_wait_cycles":18688}
I20260923 20:47:02.864509 12809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2192b9608344c08a128f99e55910ab" candidate_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "557016df63df4b268775a1ab2bc4d2cc" is_pre_election: true
I20260923 20:47:02.864964 12809 raft_consensus.cc:2454] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8b4262b58f540f2bb5432a28e9e2c3b in term 0.
I20260923 20:47:02.865166 12668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2192b9608344c08a128f99e55910ab" candidate_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67123dd3363a4d7db208c7dd58914326" is_pre_election: true
I20260923 20:47:02.865407 12668 raft_consensus.cc:2454] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8b4262b58f540f2bb5432a28e9e2c3b in term 0.
I20260923 20:47:02.865662 12462 leader_election.cc:302] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 557016df63df4b268775a1ab2bc4d2cc, c8b4262b58f540f2bb5432a28e9e2c3b; no voters:
I20260923 20:47:02.866087 12962 raft_consensus.cc:2781] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:02.866226 12962 raft_consensus.cc:482] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:02.866354 12962 raft_consensus.cc:3037] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:02.869387 12962 raft_consensus.cc:504] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.870381 12962 leader_election.cc:288] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [CANDIDATE]: Term 1 election: Requested vote from peers 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615), 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759)
I20260923 20:47:02.870795 12809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2192b9608344c08a128f99e55910ab" candidate_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "557016df63df4b268775a1ab2bc4d2cc"
I20260923 20:47:02.870805 12668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2192b9608344c08a128f99e55910ab" candidate_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67123dd3363a4d7db208c7dd58914326"
I20260923 20:47:02.871007 12809 raft_consensus.cc:3037] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:02.871016 12668 raft_consensus.cc:3037] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:02.873275 12668 raft_consensus.cc:2454] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8b4262b58f540f2bb5432a28e9e2c3b in term 1.
I20260923 20:47:02.873311 12809 raft_consensus.cc:2454] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8b4262b58f540f2bb5432a28e9e2c3b in term 1.
I20260923 20:47:02.873705 12462 leader_election.cc:302] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 557016df63df4b268775a1ab2bc4d2cc, c8b4262b58f540f2bb5432a28e9e2c3b; no voters:
I20260923 20:47:02.874033 12962 raft_consensus.cc:2781] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:02.874455 12962 raft_consensus.cc:686] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 LEADER]: Becoming Leader. State: Replica: c8b4262b58f540f2bb5432a28e9e2c3b, State: Running, Role: LEADER
I20260923 20:47:02.874882 12962 consensus_queue.cc:237] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } }
I20260923 20:47:02.875684 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.875237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:53047)
0923 20:47:02.875386 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.875526 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 20:47:02.878473 12393 catalog_manager.cc:5495] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129). New cstate: current_term: 1 leader_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b4262b58f540f2bb5432a28e9e2c3b" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 36811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 } health_report { overall_health: UNKNOWN } } }
I20260923 20:47:02.925894 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.919276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.5.71.129:36811)
0923 20:47:02.919498 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.919503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.919524 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:47:02.919743 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.919747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.919915 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.920067 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:02.920534 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.921360 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.923532 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.924094 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.924217 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:02.924718 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:02.925296 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:02.925406 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:02.925544 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 20:47:02.925613 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260923 20:47:02.930645 12668 raft_consensus.cc:1261] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Refusing update from remote peer c8b4262b58f540f2bb5432a28e9e2c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:02.931114 12809 raft_consensus.cc:1261] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Refusing update from remote peer c8b4262b58f540f2bb5432a28e9e2c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:02.931483 12965 consensus_queue.cc:1035] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "67123dd3363a4d7db208c7dd58914326" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:02.932097 12962 consensus_queue.cc:1035] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "557016df63df4b268775a1ab2bc4d2cc" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:02.934983 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.934395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:47339)
0923 20:47:02.934635 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.934759 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 20:47:02.935288 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.934727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:54761)
0923 20:47:02.934973 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.935064 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260923 20:47:02.939195 12968 mvcc.cc:205] Tried to move back new op lower bound from 7332644548316983296 to 7332644548099842048. Current Snapshot: MvccSnapshot[applied={T|T < 7332644548316983296}]
I20260923 20:47:02.941733 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.941240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.130:56507)
0923 20:47:02.941473 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.941561 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260923 20:47:02.943858 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.943042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:58385)
0923 20:47:02.943390 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.943527 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260923 20:47:02.945170 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.944664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.129:47935)
0923 20:47:02.944874 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.944990 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 20:47:02.947260 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.946739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.5.71.131:35371)
0923 20:47:02.946963 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:02.947061 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260923 20:47:02.966293 12393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:02.966611 12393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:02.973538 12393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0832bd8d4464f52b4c15575cd4a6d6a: Sending DeleteTablet for 3 replicas of tablet 8c2192b9608344c08a128f99e55910ab
I20260923 20:47:02.975142 12648 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c2192b9608344c08a128f99e55910ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:02 UTC) from {username='oryx', principal='oryx/127.5.71.188@KRBTEST.COM'} at 127.0.0.1:53586
I20260923 20:47:02.975987 12789 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c2192b9608344c08a128f99e55910ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:02 UTC) from {username='oryx', principal='oryx/127.5.71.188@KRBTEST.COM'} at 127.0.0.1:51710
I20260923 20:47:02.976061 12977 tablet_replica.cc:330] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: stopping tablet replica
I20260923 20:47:02.976236 12508 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c2192b9608344c08a128f99e55910ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:02 UTC) from {username='oryx', principal='oryx/127.5.71.188@KRBTEST.COM'} at 127.0.0.1:37078
I20260923 20:47:02.976349 12977 raft_consensus.cc:2229] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:02.976780 12977 raft_consensus.cc:2258] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:02.978452 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 12366
I20260923 20:47:02.981051 12977 ts_tablet_manager.cc:1905] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:02.988408 12979 tablet_replica.cc:330] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: stopping tablet replica
I20260923 20:47:02.988665 12979 raft_consensus.cc:2229] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:02.989473 12979 raft_consensus.cc:2258] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:02.990753 12979 ts_tablet_manager.cc:1905] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:02.991552 12977 ts_tablet_manager.cc:1918] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:02.991779 12977 log.cc:1196] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/8c2192b9608344c08a128f99e55910ab
I20260923 20:47:02.992404 12977 ts_tablet_manager.cc:1939] T 8c2192b9608344c08a128f99e55910ab P 67123dd3363a4d7db208c7dd58914326: Deleting consensus metadata
I20260923 20:47:02.993851 12381 catalog_manager.cc:4841] TS 67123dd3363a4d7db208c7dd58914326 (127.5.71.130:40759): tablet 8c2192b9608344c08a128f99e55910ab (table test-table [id=560f8d3e7e5a404f890f9abeb3eac6bb]) successfully deleted
I20260923 20:47:02.997501 12979 ts_tablet_manager.cc:1918] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:02.997678 12979 log.cc:1196] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/8c2192b9608344c08a128f99e55910ab
I20260923 20:47:02.998209 12979 ts_tablet_manager.cc:1939] T 8c2192b9608344c08a128f99e55910ab P c8b4262b58f540f2bb5432a28e9e2c3b: Deleting consensus metadata
I20260923 20:47:02.999521 12381 catalog_manager.cc:4841] TS c8b4262b58f540f2bb5432a28e9e2c3b (127.5.71.129:36811): tablet 8c2192b9608344c08a128f99e55910ab (table test-table [id=560f8d3e7e5a404f890f9abeb3eac6bb]) successfully deleted
I20260923 20:47:03.000082 12978 tablet_replica.cc:330] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: stopping tablet replica
I20260923 20:47:03.000325 12978 raft_consensus.cc:2229] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:03.000635 12978 raft_consensus.cc:2258] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:03.001667 12978 ts_tablet_manager.cc:1905] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:03.006326 12978 ts_tablet_manager.cc:1918] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:03.006487 12978 log.cc:1196] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/8c2192b9608344c08a128f99e55910ab
I20260923 20:47:03.006878 12978 ts_tablet_manager.cc:1939] T 8c2192b9608344c08a128f99e55910ab P 557016df63df4b268775a1ab2bc4d2cc: Deleting consensus metadata
I20260923 20:47:03.008117 12379 catalog_manager.cc:4841] TS 557016df63df4b268775a1ab2bc4d2cc (127.5.71.131:46615): tablet 8c2192b9608344c08a128f99e55910ab (table test-table [id=560f8d3e7e5a404f890f9abeb3eac6bb]) successfully deleted
I20260923 20:47:03.010526 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:02.980106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.5.71.129:36811)
0923 20:47:02.980345 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:02.980355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:02.988583 (+ 8228us) server_negotiation.cc:407] Connection header received
0923 20:47:02.989671 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:02.989677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:02.989790 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:02.989889 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:02.993958 (+ 4069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.995251 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.997270 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:02.998090 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:02.998259 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:03.002347 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:03.002368 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:03.002391 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:03.002443 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:03.007129 (+ 4686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:03.008076 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:03.008087 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:03.008091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:03.008299 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:03.008732 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:03.008737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:03.008740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:03.009227 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:03.009543 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:03.009859 (+ 316us) server_negotiation.cc:299] Negotiation successful
0923 20:47:03.010118 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:47:03.011201 12568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:03.116940 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 12366
I20260923 20:47:03.135535 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 12585
I20260923 20:47:03.138217 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:03.137824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43818)
0923 20:47:03.137982 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:03.138077 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 20:47:03.139256 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:03.138857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40203 (local address 127.0.0.1:43830)
0923 20:47:03.139032 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:03.139122 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260923 20:47:03.152335 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:03.136409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.5.71.130:40759)
0923 20:47:03.136626 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:03.136631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:03.136649 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:47:03.136842 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:03.136846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:03.136908 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:03.136997 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:03.138805 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:03.139719 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:03.142038 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:03.142578 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:03.142723 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:03.146048 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:03.146064 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:03.146083 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:03.146133 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:03.149397 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:03.150244 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:03.150256 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:03.150260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:03.150402 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:03.150756 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:03.150759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:03.150762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:03.151129 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:03.151413 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:03.151686 (+ 273us) server_negotiation.cc:299] Negotiation successful
0923 20:47:03.152001 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260923 20:47:03.153156 12708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:03.249056 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 12585
I20260923 20:47:03.268203 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 12726
I20260923 20:47:03.282459 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:03.269084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.5.71.131:46615)
0923 20:47:03.269263 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:03.269270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:03.269386 (+ 116us) server_negotiation.cc:407] Connection header received
0923 20:47:03.269598 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:03.269604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:03.269701 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:03.269794 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:03.271135 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:03.272037 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:03.273329 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:03.273738 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:03.273891 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:03.276779 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:03.276794 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:03.276813 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:03.276849 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:03.279652 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:03.280369 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:03.280378 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:03.280381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:03.280517 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:03.280912 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:03.280917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:03.280921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:03.281292 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:03.281614 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:03.281889 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 20:47:03.282117 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 20:47:03.283205 12849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:03.405441 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 12726
I20260923 20:47:03.427359 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 12055
I20260923 20:47:03.428537 12228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:03.546821 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 12055
I20260923 20:47:03.568341 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 12156
I20260923 20:47:03.569552 12424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:03.691885 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 12156
2026-09-23T20:47:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10036 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:47:03 dist-test-slave-dn29 krb5kdc[12988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:47:03 dist-test-slave-dn29 krb5kdc[12988](info): set up 2 sockets
Sep 23 20:47:03 dist-test-slave-dn29 krb5kdc[12988](info): commencing operation
krb5kdc: starting...
W20260923 20:47:04.809206 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:47:04 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab.
Entry for principal oryx/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab.
I20260923 20:47:05.045326 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45149
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab
--principal=oryx/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:05.378923 13001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:05.379251 13001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:05.379420 13001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:05.386904 13001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:05.387019 13001 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 20:47:05.387087 13001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:05.387181 13001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:05.387266 13001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:05.387379 13001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:05.397560 13001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.190.keytab
--principal=oryx/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:05.398979 13001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:05.400550 13001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:05.413102 13007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:05.413039 13006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:05.413153 13009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:05.414419 13001 server_base.cc:1034] running on GCE node
I20260923 20:47:05.415180 13001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:05.416807 13001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:05.418131 13001 hybrid_clock.cc:648] HybridClock initialized: now 1790196425418026 us; error 47 us; skew 500 ppm
I20260923 20:47:05.418542 13001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:05 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196425, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:05.422503 13001 init.cc:377] Logged in from keytab as oryx/127.5.71.190@KRBTEST.COM (short username oryx)
I20260923 20:47:05.424402 13001 webserver.cc:460] Webserver started at http://127.5.71.190:33671/ using document root <none> and password file <none>
I20260923 20:47:05.425238 13001 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:05.425390 13001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:05.425757 13001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:05.429041 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "a20db410821a42c58c37307c648d768b"
format_stamp: "Formatted at 2026-09-23 20:47:05 on dist-test-slave-dn29"
server_key: "55c66b5eb481225286466e4477d36a1a"
server_key_iv: "94a26213c563a8d32cb5951bebe83ade"
server_key_version: "encryptionkey@0"
I20260923 20:47:05.429878 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "a20db410821a42c58c37307c648d768b"
format_stamp: "Formatted at 2026-09-23 20:47:05 on dist-test-slave-dn29"
server_key: "55c66b5eb481225286466e4477d36a1a"
server_key_iv: "94a26213c563a8d32cb5951bebe83ade"
server_key_version: "encryptionkey@0"
I20260923 20:47:05.435222 13001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:47:05.439033 13016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:05.440521 13001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:47:05.440714 13001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "a20db410821a42c58c37307c648d768b"
format_stamp: "Formatted at 2026-09-23 20:47:05 on dist-test-slave-dn29"
server_key: "55c66b5eb481225286466e4477d36a1a"
server_key_iv: "94a26213c563a8d32cb5951bebe83ade"
server_key_version: "encryptionkey@0"
I20260923 20:47:05.440946 13001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:05.465330 13001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:05.470137 13001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:05.470438 13001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:05.494029 13001 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45149
I20260923 20:47:05.494055 13077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45149 every 8 connection(s)
I20260923 20:47:05.495543 13001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:47:05.498461 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13001
I20260923 20:47:05.498699 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:47:05.499579 5406 external_mini_cluster.cc:1428] Setting key 7fec41749eab0878ac6c446e5df94030
I20260923 20:47:05.500923 13078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:47:05 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:05.513835 13078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Bootstrap starting.
I20260923 20:47:05.518152 13078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:05.519611 13078 log.cc:824] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:05.523546 13078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: No bootstrap required, opened a new log
I20260923 20:47:05.524817 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:05.501916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.5.71.190:45149)
0923 20:47:05.502794 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:05.502807 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:05.502848 (+ 41us) server_negotiation.cc:407] Connection header received
0923 20:47:05.504612 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:05.504634 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:05.505053 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:05.505481 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:05.506892 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:05.508386 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:05.509618 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:05.510310 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:05.510523 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:05.514910 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:05.514922 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:05.514943 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:05.515002 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:05.519376 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:05.520387 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:05.520395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:05.520405 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:05.520582 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:05.521149 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:05.521155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:05.521158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:05.521999 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:05.522324 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:05.522755 (+ 431us) server_negotiation.cc:299] Negotiation successful
0923 20:47:05.523261 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":355,"threads_started":1}
I20260923 20:47:05.532847 13078 raft_consensus.cc:348] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20db410821a42c58c37307c648d768b" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45149 } }
I20260923 20:47:05.533241 13078 raft_consensus.cc:374] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:05.533320 13078 raft_consensus.cc:729] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a20db410821a42c58c37307c648d768b, State: Initialized, Role: FOLLOWER
I20260923 20:47:05.533923 13078 consensus_queue.cc:260] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20db410821a42c58c37307c648d768b" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45149 } }
I20260923 20:47:05.534241 13078 raft_consensus.cc:388] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:47:05.534443 13078 raft_consensus.cc:482] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:47:05.534693 13078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:05.537222 13078 raft_consensus.cc:504] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20db410821a42c58c37307c648d768b" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45149 } }
I20260923 20:47:05.537770 13078 leader_election.cc:302] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a20db410821a42c58c37307c648d768b; no voters:
I20260923 20:47:05.538520 13078 leader_election.cc:288] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:47:05.538666 13083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:05.539968 13083 raft_consensus.cc:686] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [term 1 LEADER]: Becoming Leader. State: Replica: a20db410821a42c58c37307c648d768b, State: Running, Role: LEADER
I20260923 20:47:05.540566 13083 consensus_queue.cc:237] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20db410821a42c58c37307c648d768b" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45149 } }
I20260923 20:47:05.541049 13078 sys_catalog.cc:564] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:47:05.543460 13085 sys_catalog.cc:455] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a20db410821a42c58c37307c648d768b. Latest consensus state: current_term: 1 leader_uuid: "a20db410821a42c58c37307c648d768b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20db410821a42c58c37307c648d768b" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45149 } } }
I20260923 20:47:05.543531 13084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a20db410821a42c58c37307c648d768b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a20db410821a42c58c37307c648d768b" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45149 } } }
I20260923 20:47:05.543865 13085 sys_catalog.cc:458] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:05.543990 13084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:05.545735 13090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:05.550899 13090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:05.561062 13090 catalog_manager.cc:1347] Generated new cluster ID: b03ec7bfcd1b4e4fa732fcb9e25505b7
I20260923 20:47:05.561192 13090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:05.591866 13090 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:47:05.593811 13090 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:05.602537 13090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Generated new TSK 0
I20260923 20:47:05.603713 13090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab.
Entry for principal oryx/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab.
I20260923 20:47:05.659190 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45149
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:05.993768 13104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:05.994076 13104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:05.994172 13104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:06.001748 13104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:06.001880 13104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:06.001971 13104 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:47:06.002051 13104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:47:06.012503 13104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:06.014027 13104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:06.015424 13104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:06.030565 13109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:06.030483 13112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:06.030458 13110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:06.031481 13104 server_base.cc:1034] running on GCE node
I20260923 20:47:06.032186 13104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:06.033195 13104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:06.034427 13104 hybrid_clock.cc:648] HybridClock initialized: now 1790196426034369 us; error 27 us; skew 500 ppm
I20260923 20:47:06.034780 13104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:06 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196426, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:06.038283 13104 init.cc:377] Logged in from keytab as oryx/127.5.71.129@KRBTEST.COM (short username oryx)
I20260923 20:47:06.039937 13104 webserver.cc:460] Webserver started at http://127.5.71.129:42143/ using document root <none> and password file <none>
I20260923 20:47:06.040661 13104 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:06.040779 13104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:06.041164 13104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:06.044490 13104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "08bd0c233537425eba8e2bbcc426ce57"
format_stamp: "Formatted at 2026-09-23 20:47:06 on dist-test-slave-dn29"
server_key: "33ad602723024bcd8ac3d4ac18cd58bb"
server_key_iv: "c73c15bb0819bd7bf52191535d0617da"
server_key_version: "encryptionkey@0"
I20260923 20:47:06.045403 13104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "08bd0c233537425eba8e2bbcc426ce57"
format_stamp: "Formatted at 2026-09-23 20:47:06 on dist-test-slave-dn29"
server_key: "33ad602723024bcd8ac3d4ac18cd58bb"
server_key_iv: "c73c15bb0819bd7bf52191535d0617da"
server_key_version: "encryptionkey@0"
I20260923 20:47:06.051153 13104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:47:06.055073 13119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:06.056918 13104 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:47:06.057126 13104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "08bd0c233537425eba8e2bbcc426ce57"
format_stamp: "Formatted at 2026-09-23 20:47:06 on dist-test-slave-dn29"
server_key: "33ad602723024bcd8ac3d4ac18cd58bb"
server_key_iv: "c73c15bb0819bd7bf52191535d0617da"
server_key_version: "encryptionkey@0"
I20260923 20:47:06.057376 13104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:06.072928 13104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:06.078028 13104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:06.078351 13104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:06.081655 13104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:06.081797 13104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:06.081959 13104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:06.082104 13104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 23 20:47:06 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196426, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:06.110270 13128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.082334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:44847 (local address 127.5.71.190:45149)
0923 20:47:06.083537 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:06.083543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:06.084878 (+ 1335us) server_negotiation.cc:407] Connection header received
0923 20:47:06.087660 (+ 2782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:06.087669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:06.087843 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:06.088000 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:06.090890 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.092251 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.095678 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.096460 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.096633 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.102064 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:06.102074 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:06.102080 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:06.102135 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:06.105597 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.106810 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.106818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.106823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.106981 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.107527 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.107535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.107538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.107982 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:06.108116 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:06.109361 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0923 20:47:06.109695 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1068,"thread_start_us":960,"threads_started":1}
I20260923 20:47:06.110999 13129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.083466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.5.71.129:44847)
0923 20:47:06.084650 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:06.084707 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:06.087215 (+ 2508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:06.088345 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:06.088366 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:06.089383 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:06.090595 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.090627 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.092491 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.092496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.095452 (+ 2956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.095468 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.096762 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.096772 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.097190 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.098176 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:06.098220 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:06.101768 (+ 3548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:06.105782 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.105793 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.105818 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.106569 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.107135 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.107139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.107145 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.107392 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.108206 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:06.108216 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:06.108823 (+ 607us) client_negotiation.cc:769] Sending connection context
0923 20:47:06.109227 (+ 404us) client_negotiation.cc:240] Negotiation successful
0923 20:47:06.109695 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":634,"threads_started":1}
I20260923 20:47:06.137573 13104 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:37837
I20260923 20:47:06.137655 13239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:37837 every 8 connection(s)
I20260923 20:47:06.138185 13128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.118548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:32979 (local address 127.5.71.190:45149)
0923 20:47:06.118904 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:06.118910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:06.120148 (+ 1238us) server_negotiation.cc:407] Connection header received
0923 20:47:06.120471 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:06.120475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:06.120544 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:06.120658 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:06.122502 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.123736 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.126173 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.126954 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.127110 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.130739 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:06.130752 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:06.130760 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:06.130823 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:06.134340 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.135435 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.135440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.135443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.135715 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.136236 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.136242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.136246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.136755 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:06.136959 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:06.137571 (+ 612us) server_negotiation.cc:299] Negotiation successful
0923 20:47:06.137886 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:47:06.138254 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.118907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.5.71.129:32979)
0923 20:47:06.119978 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:06.120005 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:06.120294 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:06.120816 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:06.120824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:06.121654 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:06.122285 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.122304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.123925 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.123937 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.125949 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.125978 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.127260 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.127275 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.127513 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.128751 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:06.128779 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:06.130421 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:06.134529 (+ 4108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.134538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.134545 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.135292 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.135882 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.135889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.135895 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.136094 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.136935 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:06.136940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:06.137189 (+ 249us) client_negotiation.cc:769] Sending connection context
0923 20:47:06.137405 (+ 216us) client_negotiation.cc:240] Negotiation successful
0923 20:47:06.137806 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":630,"threads_started":1}
I20260923 20:47:06.139334 13104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:47:06.142081 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13104
I20260923 20:47:06.142279 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:47:06.142290 13240 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45149
I20260923 20:47:06.142711 13240 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:06.143002 5406 external_mini_cluster.cc:1428] Setting key 19874a0d092861e7a0e9fe8632e77291
I20260923 20:47:06.143455 13240 heartbeater.cc:511] Master 127.5.71.190:45149 requested a full tablet report, sending...
I20260923 20:47:06.145752 13032 ts_manager.cc:194] Registered new tserver with Master: 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837)
I20260923 20:47:06.147235 13032 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:44847
WARNING: no policy specified for oryx/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab.
Entry for principal oryx/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab.
I20260923 20:47:06.195180 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45149
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:06.552671 13244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:06.553016 13244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:06.553120 13244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:06.561551 13244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:06.561709 13244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:06.561796 13244 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:47:06.561872 13244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:47:06.572350 13244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:06.573649 13244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:06.574962 13244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:06.589368 13249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:06.589413 13250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:06.589561 13252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:06.590446 13244 server_base.cc:1034] running on GCE node
I20260923 20:47:06.591104 13244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:06.592521 13244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:06.593835 13244 hybrid_clock.cc:648] HybridClock initialized: now 1790196426593768 us; error 40 us; skew 500 ppm
I20260923 20:47:06.594182 13244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:06 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196426, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:06.597873 13244 init.cc:377] Logged in from keytab as oryx/127.5.71.130@KRBTEST.COM (short username oryx)
I20260923 20:47:06.599643 13244 webserver.cc:460] Webserver started at http://127.5.71.130:43783/ using document root <none> and password file <none>
I20260923 20:47:06.600484 13244 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:06.600582 13244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:06.600883 13244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:06.604018 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "c2e668f71bc0422cb74da107e401b435"
format_stamp: "Formatted at 2026-09-23 20:47:06 on dist-test-slave-dn29"
server_key: "292c95ec2b74892af3ede745c86d4008"
server_key_iv: "4eebce9ccf3b9bdc8833b7a10c8c2949"
server_key_version: "encryptionkey@0"
I20260923 20:47:06.604801 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "c2e668f71bc0422cb74da107e401b435"
format_stamp: "Formatted at 2026-09-23 20:47:06 on dist-test-slave-dn29"
server_key: "292c95ec2b74892af3ede745c86d4008"
server_key_iv: "4eebce9ccf3b9bdc8833b7a10c8c2949"
server_key_version: "encryptionkey@0"
I20260923 20:47:06.610531 13244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:47:06.614859 13259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:06.616650 13244 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:47:06.616890 13244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "c2e668f71bc0422cb74da107e401b435"
format_stamp: "Formatted at 2026-09-23 20:47:06 on dist-test-slave-dn29"
server_key: "292c95ec2b74892af3ede745c86d4008"
server_key_iv: "4eebce9ccf3b9bdc8833b7a10c8c2949"
server_key_version: "encryptionkey@0"
I20260923 20:47:06.617162 13244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:06.650194 13244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:06.655144 13244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:06.655514 13244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:06.658506 13244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:06.658643 13244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:06.658854 13244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:06.658946 13244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:47:06 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196426, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:06.688445 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.659466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:35913 (local address 127.5.71.190:45149)
0923 20:47:06.659941 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:06.659949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:06.661606 (+ 1657us) server_negotiation.cc:407] Connection header received
0923 20:47:06.665324 (+ 3718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:06.665333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:06.665448 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:06.665602 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:06.668453 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.670181 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.673182 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.673923 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.674084 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.680142 (+ 6058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:06.680147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:06.680151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:06.680190 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:06.683773 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.685045 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.685049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.685052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.685264 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.685953 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.685962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.685968 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.686314 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:06.686495 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:06.687589 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0923 20:47:06.687897 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":227,"threads_started":1}
I20260923 20:47:06.691440 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.660273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.5.71.130:35913)
0923 20:47:06.661378 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:06.661440 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:06.664829 (+ 3389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:06.665950 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:06.665960 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:06.667025 (+ 1065us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:06.668132 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.668156 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.670431 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.670436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.672990 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.673005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.674233 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.674242 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.674603 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.675551 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:06.675600 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:06.679861 (+ 4261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:06.684014 (+ 4153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.684032 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.684059 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.684894 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.685421 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.685433 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.685441 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.685740 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.686481 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:06.686494 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:06.687099 (+ 605us) client_negotiation.cc:769] Sending connection context
0923 20:47:06.687498 (+ 399us) client_negotiation.cc:240] Negotiation successful
0923 20:47:06.689618 (+ 2120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":465,"threads_started":1}
I20260923 20:47:06.715286 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.698365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:54617 (local address 127.5.71.190:45149)
0923 20:47:06.698574 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:06.698581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:06.699542 (+ 961us) server_negotiation.cc:407] Connection header received
0923 20:47:06.699816 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:06.699824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:06.699923 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:06.700042 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:06.701342 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.702143 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.703615 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.704182 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.704357 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.707340 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:06.707351 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:06.707357 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:06.707403 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:06.711012 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.712257 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.712266 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.712273 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.712449 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:06.713083 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:06.713092 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:06.713097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:06.713583 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:06.713770 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:06.714652 (+ 882us) server_negotiation.cc:299] Negotiation successful
0923 20:47:06.714915 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260923 20:47:06.715469 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:06.698476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.5.71.130:54617)
0923 20:47:06.699402 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:06.699426 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:06.699632 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:06.700079 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:06.700084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:06.700744 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:06.701106 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.701117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.702303 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.702309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.703450 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:06.703462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:06.704494 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:06.704504 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:06.704852 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:06.705574 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:06.705612 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:06.707058 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:06.711203 (+ 4145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.711214 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.711222 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.712094 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.712617 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:06.712622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:06.712626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:06.712841 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:06.713868 (+ 1027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:06.713876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:06.714242 (+ 366us) client_negotiation.cc:769] Sending connection context
0923 20:47:06.714557 (+ 315us) client_negotiation.cc:240] Negotiation successful
0923 20:47:06.714955 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":536,"threads_started":1}
I20260923 20:47:06.719183 13244 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38499
I20260923 20:47:06.719228 13379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38499 every 8 connection(s)
I20260923 20:47:06.720649 13244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:47:06.723337 13380 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45149
I20260923 20:47:06.723598 13380 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:06.724335 13380 heartbeater.cc:511] Master 127.5.71.190:45149 requested a full tablet report, sending...
I20260923 20:47:06.725999 13032 ts_manager.cc:194] Registered new tserver with Master: c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499)
I20260923 20:47:06.726837 13032 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:35913
I20260923 20:47:06.730607 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13244
I20260923 20:47:06.730803 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:47:06.731194 5406 external_mini_cluster.cc:1428] Setting key 0306bfc6015ea300d9c7cd6fe2476a22
WARNING: no policy specified for oryx/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab.
Entry for principal oryx/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab.
I20260923 20:47:06.784044 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45149
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:07.118021 13384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:07.118333 13384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:07.118508 13384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:07.126288 13384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:07.126406 13384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:07.126592 13384 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 20:47:07.126727 13384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:07.137269 13384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.71.190:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:07.138942 13384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:07.140826 13384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 20:47:07.150168 13240 heartbeater.cc:503] Master 127.5.71.190:45149 was elected leader, sending a full tablet report...
W20260923 20:47:07.156308 13390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:07.156368 13389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:07.156499 13392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:07.157120 13384 server_base.cc:1034] running on GCE node
I20260923 20:47:07.157776 13384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:07.158744 13384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:07.159941 13384 hybrid_clock.cc:648] HybridClock initialized: now 1790196427159865 us; error 29 us; skew 500 ppm
I20260923 20:47:07.160270 13384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196427, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:07.170328 13384 init.cc:377] Logged in from keytab as oryx/127.5.71.131@KRBTEST.COM (short username oryx)
I20260923 20:47:07.172154 13384 webserver.cc:460] Webserver started at http://127.5.71.131:42377/ using document root <none> and password file <none>
I20260923 20:47:07.172894 13384 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:07.172982 13384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:07.173249 13384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:07.176337 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "353283811d90497ca195f1eeeba48f99"
format_stamp: "Formatted at 2026-09-23 20:47:07 on dist-test-slave-dn29"
server_key: "9547347887400e88b87ed72aba717005"
server_key_iv: "757e356fafbf7b5d3299e022d9656ee1"
server_key_version: "encryptionkey@0"
I20260923 20:47:07.177100 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "353283811d90497ca195f1eeeba48f99"
format_stamp: "Formatted at 2026-09-23 20:47:07 on dist-test-slave-dn29"
server_key: "9547347887400e88b87ed72aba717005"
server_key_iv: "757e356fafbf7b5d3299e022d9656ee1"
server_key_version: "encryptionkey@0"
I20260923 20:47:07.182771 13384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:47:07.186697 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:07.188403 13384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:47:07.188577 13384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "353283811d90497ca195f1eeeba48f99"
format_stamp: "Formatted at 2026-09-23 20:47:07 on dist-test-slave-dn29"
server_key: "9547347887400e88b87ed72aba717005"
server_key_iv: "757e356fafbf7b5d3299e022d9656ee1"
server_key_version: "encryptionkey@0"
I20260923 20:47:07.188903 13384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:07.209038 13384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:07.214653 13384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:07.214931 13384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:07.218649 13384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:07.218801 13384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:07.219062 13384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:07.219192 13384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196427, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:07.243577 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.219025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:60041 (local address 127.5.71.190:45149)
0923 20:47:07.219563 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.219572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.221170 (+ 1598us) server_negotiation.cc:407] Connection header received
0923 20:47:07.223431 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.223437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.223529 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.223664 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:07.225869 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.226975 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.229732 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.230860 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.231095 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.235950 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:07.235955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:07.235958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:07.236023 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:07.239639 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.240790 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.240797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.240800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.240938 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.241601 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.241607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.241612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.241914 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:07.242072 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.242933 (+ 861us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.243149 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":290,"threads_started":1}
I20260923 20:47:07.245421 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.219886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.5.71.131:60041)
0923 20:47:07.221001 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.221041 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.223119 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.223946 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.223955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.224710 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:07.225622 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.225654 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.227144 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.227150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.229518 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.229537 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.231151 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.231157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.231379 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.232295 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:07.232346 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:07.235630 (+ 3284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:07.239921 (+ 4291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:07.239931 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:07.239959 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:07.240639 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:07.241061 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:07.241066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:07.241069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:07.241471 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:07.242047 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:07.242055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:07.242567 (+ 512us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.242842 (+ 275us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.243464 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":470,"threads_started":1}
I20260923 20:47:07.268498 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.251499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:57231 (local address 127.5.71.190:45149)
0923 20:47:07.251728 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.251736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.252595 (+ 859us) server_negotiation.cc:407] Connection header received
0923 20:47:07.252808 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.252811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.252873 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.252955 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:07.254446 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.255203 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.257814 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.258423 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.258596 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.261458 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:07.261463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:07.261467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:07.261499 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:07.265114 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.266044 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.266051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.266057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.266222 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.266720 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.266726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.266731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.267078 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:07.267248 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.267877 (+ 629us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.268131 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:47:07.268946 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.251501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.5.71.131:57231)
0923 20:47:07.252467 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.252493 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.252678 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.253026 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.253033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.253784 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:07.254263 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.254279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.255353 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.255360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.257591 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.257612 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.258656 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.258669 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.258945 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.260017 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:07.260047 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:07.261261 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:07.265319 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:07.265327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:07.265333 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:07.265907 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:07.266371 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:07.266376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:07.266383 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:07.266567 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:07.267238 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:07.267247 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:07.267514 (+ 267us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.267707 (+ 193us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.268211 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":527,"threads_started":1}
I20260923 20:47:07.283190 13384 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:38397
I20260923 20:47:07.283233 13519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:38397 every 8 connection(s)
I20260923 20:47:07.284541 13384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:07.286893 13520 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45149
I20260923 20:47:07.287093 13520 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:07.287945 13520 heartbeater.cc:511] Master 127.5.71.190:45149 requested a full tablet report, sending...
I20260923 20:47:07.288281 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13384
I20260923 20:47:07.288462 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:47:07.288982 5406 external_mini_cluster.cc:1428] Setting key bf6d1e52ad6a24a29254fd00905b5a2f
I20260923 20:47:07.289748 13032 ts_manager.cc:194] Registered new tserver with Master: 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:07.290782 13032 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:60041
I20260923 20:47:07.292322 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 20:47:07.344691 13407 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:07.333641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.5.71.190:45149)
0923 20:47:07.333864 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.333871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.334000 (+ 129us) server_negotiation.cc:407] Connection header received
0923 20:47:07.334257 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.334263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.334327 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.334437 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:07.335776 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.336530 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.337851 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.338281 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.338422 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.344448 (+ 6026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53660: BlockingRecv error: recv got EOF from 127.0.0.1:53660 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196427, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:07.361917 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.348219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.5.71.190:45149)
0923 20:47:07.348416 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.348420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.348771 (+ 351us) server_negotiation.cc:407] Connection header received
0923 20:47:07.348962 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.348968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.349041 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.349152 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:07.350283 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.351009 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.352316 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.352777 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.352916 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.356754 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:07.356760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:07.356763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:07.356799 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:07.359188 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.359925 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.359930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.359934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.360068 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.360456 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.360462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.360466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.360734 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:07.361080 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.361366 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.361601 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:47:07.366277 13033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:07.373359 13033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:07.409622 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.387065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.5.71.129:37837)
0923 20:47:07.400788 (+ 13723us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.400798 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.400835 (+ 37us) server_negotiation.cc:407] Connection header received
0923 20:47:07.400970 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.400983 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.401284 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.401553 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.402910 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.404677 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.407054 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.408398 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.408559 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.408815 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.409099 (+ 284us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.409273 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13546,"thread_start_us":14701,"threads_started":1}
I20260923 20:47:07.409672 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.386760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:37837 (local address 127.0.0.1:57306)
0923 20:47:07.399865 (+ 13105us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.399923 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.400187 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.402078 (+ 1891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.402087 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.402125 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.402716 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.402735 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.404838 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.404845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.406845 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.406860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.408625 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.408633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.408844 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.408901 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.409053 (+ 152us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.409280 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12743,"mutex_wait_us":38,"thread_start_us":13402,"threads_started":1}
I20260923 20:47:07.410845 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.386966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38397 (local address 127.0.0.1:40330)
0923 20:47:07.400457 (+ 13491us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.400476 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.400594 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.402003 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.402015 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.402059 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.403132 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.403147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.405144 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.405182 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.407485 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.407506 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.409982 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.409992 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.410212 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.410249 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.410350 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.410480 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13346,"mutex_wait_us":4,"thread_start_us":12375,"threads_started":1}
I20260923 20:47:07.410907 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.387456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40330 (local address 127.5.71.131:38397)
0923 20:47:07.399780 (+ 12324us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.399812 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.400736 (+ 924us) server_negotiation.cc:407] Connection header received
0923 20:47:07.400852 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.400861 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.401317 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.401553 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.403415 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.404976 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.407693 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.409284 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.409397 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.409616 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.410421 (+ 805us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.410591 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12127,"thread_start_us":12353,"threads_started":1}
I20260923 20:47:07.413056 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.400643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38499 (local address 127.0.0.1:49548)
0923 20:47:07.403481 (+ 2838us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.403497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.403697 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.404594 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.404597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.404618 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.405032 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.405042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.406916 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.406921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.409157 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.409172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.412232 (+ 3060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.412239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.412413 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.412444 (+ 31us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.412537 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.412686 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2715,"thread_start_us":2572,"threads_started":1}
I20260923 20:47:07.413569 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.400855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.5.71.130:38499)
0923 20:47:07.402966 (+ 2111us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.402978 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.403586 (+ 608us) server_negotiation.cc:407] Connection header received
0923 20:47:07.403908 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.403916 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.404197 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.404370 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.405239 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.406747 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.409382 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.410768 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.410915 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.412056 (+ 1141us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.413009 (+ 953us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.413282 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1897,"thread_start_us":1759,"threads_started":1}
I20260923 20:47:07.414466 13170 tablet_service.cc:1467] Processing CreateTablet for tablet d73343440aee4212bf04694ad282af93 (DEFAULT_TABLE table=test-table [id=46d4585826c7456c8e8a8f3e874c7c4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:07.414712 13450 tablet_service.cc:1467] Processing CreateTablet for tablet d73343440aee4212bf04694ad282af93 (DEFAULT_TABLE table=test-table [id=46d4585826c7456c8e8a8f3e874c7c4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:07.416101 13170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73343440aee4212bf04694ad282af93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.416203 13450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73343440aee4212bf04694ad282af93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.417150 13310 tablet_service.cc:1467] Processing CreateTablet for tablet d73343440aee4212bf04694ad282af93 (DEFAULT_TABLE table=test-table [id=46d4585826c7456c8e8a8f3e874c7c4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:07.418529 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d73343440aee4212bf04694ad282af93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.428665 13544 tablet_bootstrap.cc:492] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Bootstrap starting.
I20260923 20:47:07.430751 13545 tablet_bootstrap.cc:492] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Bootstrap starting.
I20260923 20:47:07.433202 13544 tablet_bootstrap.cc:654] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.434312 13545 tablet_bootstrap.cc:654] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.434600 13544 log.cc:824] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:07.435730 13545 log.cc:824] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:07.436777 13546 tablet_bootstrap.cc:492] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Bootstrap starting.
I20260923 20:47:07.438244 13544 tablet_bootstrap.cc:492] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: No bootstrap required, opened a new log
I20260923 20:47:07.438557 13544 ts_tablet_manager.cc:1397] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260923 20:47:07.438834 13545 tablet_bootstrap.cc:492] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: No bootstrap required, opened a new log
I20260923 20:47:07.439078 13545 ts_tablet_manager.cc:1397] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260923 20:47:07.439960 13546 tablet_bootstrap.cc:654] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.441056 13546 log.cc:824] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:07.443616 13546 tablet_bootstrap.cc:492] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: No bootstrap required, opened a new log
I20260923 20:47:07.443933 13546 ts_tablet_manager.cc:1397] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260923 20:47:07.449841 13544 raft_consensus.cc:348] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.449843 13545 raft_consensus.cc:348] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.450328 13544 raft_consensus.cc:374] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.450340 13545 raft_consensus.cc:374] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.450438 13544 raft_consensus.cc:729] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bd0c233537425eba8e2bbcc426ce57, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.450438 13545 raft_consensus.cc:729] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353283811d90497ca195f1eeeba48f99, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.451112 13544 consensus_queue.cc:260] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.451115 13545 consensus_queue.cc:260] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.452822 13520 heartbeater.cc:503] Master 127.5.71.190:45149 was elected leader, sending a full tablet report...
I20260923 20:47:07.453207 13545 ts_tablet_manager.cc:1428] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260923 20:47:07.453258 13546 raft_consensus.cc:348] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.453704 13546 raft_consensus.cc:374] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.453794 13546 raft_consensus.cc:729] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.454530 13546 consensus_queue.cc:260] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.455588 13380 heartbeater.cc:503] Master 127.5.71.190:45149 was elected leader, sending a full tablet report...
I20260923 20:47:07.456485 13544 ts_tablet_manager.cc:1428] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260923 20:47:07.456954 13552 raft_consensus.cc:482] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:07.457283 13552 raft_consensus.cc:504] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.457813 13546 ts_tablet_manager.cc:1428] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260923 20:47:07.460376 13552 leader_election.cc:288] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:07.467164 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.460038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33013 (local address 127.5.71.129:37837)
0923 20:47:07.460776 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.460784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.461174 (+ 390us) server_negotiation.cc:407] Connection header received
0923 20:47:07.461459 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.461466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.461562 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.461628 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.462362 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.463218 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.465017 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.466221 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.466369 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.466420 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.466889 (+ 469us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.466983 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593}
I20260923 20:47:07.467859 13553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.459576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:37837 (local address 127.5.71.130:33013)
0923 20:47:07.461069 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.461096 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.461243 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.461807 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.461811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.461831 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.462159 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.462169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.463356 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.463359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.464858 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.464869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.466486 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.466494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.466706 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.466743 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.466867 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.467019 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"thread_start_us":1344,"threads_started":1}
I20260923 20:47:07.468168 13554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.460961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38397 (local address 127.5.71.130:39805)
0923 20:47:07.461862 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.461884 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.462074 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.462599 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.462602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.462628 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.462898 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.462905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.463931 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.463938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.465533 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.465546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.467364 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.467370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.467535 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.467557 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.467631 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.467718 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":1253,"threads_started":1}
I20260923 20:47:07.468155 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.460867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:39805 (local address 127.5.71.131:38397)
0923 20:47:07.461082 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.461088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.461959 (+ 871us) server_negotiation.cc:407] Connection header received
0923 20:47:07.462296 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.462303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.462375 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.462447 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.463045 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.463730 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.465827 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.467140 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.467324 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.467422 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.467692 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.467881 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 20:47:07.468963 13194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73343440aee4212bf04694ad282af93" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57" is_pre_election: true
I20260923 20:47:07.469121 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73343440aee4212bf04694ad282af93" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99" is_pre_election: true
I20260923 20:47:07.469471 13194 raft_consensus.cc:2454] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 0.
I20260923 20:47:07.469708 13475 raft_consensus.cc:2454] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 0.
I20260923 20:47:07.470242 13262 leader_election.cc:302] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:07.470698 13552 raft_consensus.cc:2781] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:07.470857 13552 raft_consensus.cc:482] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:07.470965 13552 raft_consensus.cc:3037] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Advancing to term 1
W20260923 20:47:07.473127 13381 tablet.cc:2369] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:47:07.473560 13552 raft_consensus.cc:504] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.474395 13552 leader_election.cc:288] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 election: Requested vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:07.474885 13194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73343440aee4212bf04694ad282af93" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57"
I20260923 20:47:07.475023 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d73343440aee4212bf04694ad282af93" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99"
I20260923 20:47:07.475119 13194 raft_consensus.cc:3037] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.475261 13475 raft_consensus.cc:3037] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.477579 13475 raft_consensus.cc:2454] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 1.
I20260923 20:47:07.477902 13194 raft_consensus.cc:2454] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 1.
I20260923 20:47:07.478013 13263 leader_election.cc:302] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353283811d90497ca195f1eeeba48f99, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:07.478385 13552 raft_consensus.cc:2781] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:07.478858 13552 raft_consensus.cc:686] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 1 LEADER]: Becoming Leader. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Running, Role: LEADER
I20260923 20:47:07.479373 13552 consensus_queue.cc:237] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } }
I20260923 20:47:07.484911 13032 catalog_manager.cc:5495] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2e668f71bc0422cb74da107e401b435 (127.5.71.130). New cstate: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } health_report { overall_health: HEALTHY } } }
I20260923 20:47:07.503720 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.498417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.5.71.130:38499)
0923 20:47:07.498630 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.498636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.498657 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:47:07.498822 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.498826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.498905 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.499059 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:07.499421 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.500225 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.501535 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.501948 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.502079 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.502545 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.503114 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.503277 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.503363 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.503443 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260923 20:47:07.512341 13194 raft_consensus.cc:1261] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Refusing update from remote peer c2e668f71bc0422cb74da107e401b435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:07.512341 13475 raft_consensus.cc:1261] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Refusing update from remote peer c2e668f71bc0422cb74da107e401b435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:07.513346 13552 consensus_queue.cc:1035] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:07.513672 13555 consensus_queue.cc:1035] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:07.525053 13558 mvcc.cc:205] Tried to move back new op lower bound from 7332644567080468480 to 7332644566959960064. Current Snapshot: MvccSnapshot[applied={T|T < 7332644567080468480}]
I20260923 20:47:07.526975 13559 mvcc.cc:205] Tried to move back new op lower bound from 7332644567080468480 to 7332644566959960064. Current Snapshot: MvccSnapshot[applied={T|T < 7332644567080468480}]
I20260923 20:47:07.542030 13032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:07.542356 13032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:07.546938 13032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Sending DeleteTablet for 3 replicas of tablet d73343440aee4212bf04694ad282af93
I20260923 20:47:07.548373 13310 tablet_service.cc:1514] Processing DeleteTablet for tablet d73343440aee4212bf04694ad282af93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:07 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:49548
I20260923 20:47:07.548444 13170 tablet_service.cc:1514] Processing DeleteTablet for tablet d73343440aee4212bf04694ad282af93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:07 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57306
I20260923 20:47:07.548662 13450 tablet_service.cc:1514] Processing DeleteTablet for tablet d73343440aee4212bf04694ad282af93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:07 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:40330
I20260923 20:47:07.552065 13032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:07.553637 13032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:07.563073 13570 tablet_replica.cc:330] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: stopping tablet replica
I20260923 20:47:07.564365 13570 raft_consensus.cc:2229] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:07.565464 13569 tablet_replica.cc:330] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: stopping tablet replica
I20260923 20:47:07.566201 13570 raft_consensus.cc:2258] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:07.566855 13569 raft_consensus.cc:2229] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:07.567010 13309 tablet_service.cc:1467] Processing CreateTablet for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd (DEFAULT_TABLE table=test-table [id=d50d0e37651c4cabaf0118570cf92ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:07.567687 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.569675 13570 ts_tablet_manager.cc:1905] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:07.570295 13569 raft_consensus.cc:2258] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:07.572781 13569 ts_tablet_manager.cc:1905] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:07.572695 13449 tablet_service.cc:1467] Processing CreateTablet for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd (DEFAULT_TABLE table=test-table [id=d50d0e37651c4cabaf0118570cf92ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:07.573304 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.578246 13546 tablet_bootstrap.cc:492] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Bootstrap starting.
I20260923 20:47:07.580147 13546 tablet_bootstrap.cc:654] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.581774 13545 tablet_bootstrap.cc:492] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Bootstrap starting.
I20260923 20:47:07.583766 13545 tablet_bootstrap.cc:654] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.565693 13169 tablet_service.cc:1467] Processing CreateTablet for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd (DEFAULT_TABLE table=test-table [id=d50d0e37651c4cabaf0118570cf92ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:07.584456 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.592130 13546 tablet_bootstrap.cc:492] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: No bootstrap required, opened a new log
I20260923 20:47:07.592334 13546 ts_tablet_manager.cc:1397] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260923 20:47:07.593410 13546 raft_consensus.cc:348] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.593708 13546 raft_consensus.cc:374] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.593853 13546 raft_consensus.cc:729] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.594241 13546 consensus_queue.cc:260] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.594964 13546 ts_tablet_manager.cc:1428] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 20:47:07.595762 13568 tablet_replica.cc:330] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: stopping tablet replica
I20260923 20:47:07.596305 13568 raft_consensus.cc:2229] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:07.597052 13568 raft_consensus.cc:2258] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:07.599272 13568 ts_tablet_manager.cc:1905] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:07.599633 13544 tablet_bootstrap.cc:492] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Bootstrap starting.
I20260923 20:47:07.601441 13544 tablet_bootstrap.cc:654] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.606305 13570 ts_tablet_manager.cc:1918] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:07.606403 13545 tablet_bootstrap.cc:492] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: No bootstrap required, opened a new log
I20260923 20:47:07.606495 13570 log.cc:1196] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/d73343440aee4212bf04694ad282af93
I20260923 20:47:07.606546 13545 ts_tablet_manager.cc:1397] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.008s
I20260923 20:47:07.607133 13570 ts_tablet_manager.cc:1939] T d73343440aee4212bf04694ad282af93 P 353283811d90497ca195f1eeeba48f99: Deleting consensus metadata
I20260923 20:47:07.607517 13545 raft_consensus.cc:348] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.607758 13545 raft_consensus.cc:374] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.607877 13545 raft_consensus.cc:729] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353283811d90497ca195f1eeeba48f99, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.608191 13545 consensus_queue.cc:260] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.609233 13020 catalog_manager.cc:4841] TS 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397): tablet d73343440aee4212bf04694ad282af93 (table test-table [id=46d4585826c7456c8e8a8f3e874c7c4d]) successfully deleted
I20260923 20:47:07.609468 13545 ts_tablet_manager.cc:1428] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 20:47:07.610481 13544 tablet_bootstrap.cc:492] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: No bootstrap required, opened a new log
I20260923 20:47:07.610724 13544 ts_tablet_manager.cc:1397] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260923 20:47:07.611094 13568 ts_tablet_manager.cc:1918] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:07.611279 13568 log.cc:1196] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/d73343440aee4212bf04694ad282af93
I20260923 20:47:07.611869 13544 raft_consensus.cc:348] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.612207 13544 raft_consensus.cc:374] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.612368 13544 raft_consensus.cc:729] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bd0c233537425eba8e2bbcc426ce57, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.612785 13544 consensus_queue.cc:260] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.613257 13568 ts_tablet_manager.cc:1939] T d73343440aee4212bf04694ad282af93 P c2e668f71bc0422cb74da107e401b435: Deleting consensus metadata
I20260923 20:47:07.615667 13020 catalog_manager.cc:4841] TS c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499): tablet d73343440aee4212bf04694ad282af93 (table test-table [id=46d4585826c7456c8e8a8f3e874c7c4d]) successfully deleted
I20260923 20:47:07.616529 13544 ts_tablet_manager.cc:1428] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260923 20:47:07.618104 13569 ts_tablet_manager.cc:1918] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:07.618284 13569 log.cc:1196] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/d73343440aee4212bf04694ad282af93
I20260923 20:47:07.618848 13569 ts_tablet_manager.cc:1939] T d73343440aee4212bf04694ad282af93 P 08bd0c233537425eba8e2bbcc426ce57: Deleting consensus metadata
I20260923 20:47:07.620370 13019 catalog_manager.cc:4841] TS 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837): tablet d73343440aee4212bf04694ad282af93 (table test-table [id=46d4585826c7456c8e8a8f3e874c7c4d]) successfully deleted
W20260923 20:47:07.642705 13241 tablet.cc:2369] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:47:07.647533 13550 raft_consensus.cc:482] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:07.647940 13550 raft_consensus.cc:504] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.649746 13550 leader_election.cc:288] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499)
I20260923 20:47:07.658459 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.650674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:33037 (local address 127.5.71.130:38499)
0923 20:47:07.650888 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.650899 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.650925 (+ 26us) server_negotiation.cc:407] Connection header received
0923 20:47:07.651160 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.651167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.651250 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.651366 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.652132 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.653504 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.655932 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.657300 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.657481 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.657565 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.657896 (+ 331us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.658030 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 20:47:07.658532 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.650216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38499 (local address 127.5.71.131:33037)
0923 20:47:07.650857 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.650876 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.650995 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.651470 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.651475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.651508 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.651943 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.651959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.653705 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.653713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.655694 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.655713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.657513 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.657520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.657703 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.657741 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.657867 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.658020 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":76,"thread_start_us":960,"threads_started":1}
I20260923 20:47:07.659034 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.650074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:37837 (local address 127.5.71.131:50219)
0923 20:47:07.650307 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.650337 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.650475 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.651620 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.651630 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.651675 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:07.652023 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.652037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.653689 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.653696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.656214 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.656231 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.658206 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.658213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.658429 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.658471 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.658573 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.658737 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":199}
I20260923 20:47:07.659840 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.650886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:50219 (local address 127.5.71.129:37837)
0923 20:47:07.651072 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.651078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.651098 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:47:07.651178 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.651184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.651264 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.651404 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:07.652207 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.653504 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.656426 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.657850 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.658031 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.658107 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.659345 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.659520 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 20:47:07.660326 13330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d8b9eb7b141c29d9395a3c5a055cd" candidate_uuid: "353283811d90497ca195f1eeeba48f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e668f71bc0422cb74da107e401b435" is_pre_election: true
I20260923 20:47:07.660492 13193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d8b9eb7b141c29d9395a3c5a055cd" candidate_uuid: "353283811d90497ca195f1eeeba48f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57" is_pre_election: true
I20260923 20:47:07.660821 13193 raft_consensus.cc:2454] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353283811d90497ca195f1eeeba48f99 in term 0.
I20260923 20:47:07.660852 13330 raft_consensus.cc:2454] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 353283811d90497ca195f1eeeba48f99 in term 0.
I20260923 20:47:07.661548 13402 leader_election.cc:302] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, 353283811d90497ca195f1eeeba48f99, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:07.662118 13550 raft_consensus.cc:2781] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:07.662298 13550 raft_consensus.cc:482] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:07.662441 13550 raft_consensus.cc:3037] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.664876 13550 raft_consensus.cc:504] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.665624 13550 leader_election.cc:288] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [CANDIDATE]: Term 1 election: Requested vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499)
I20260923 20:47:07.666193 13193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d8b9eb7b141c29d9395a3c5a055cd" candidate_uuid: "353283811d90497ca195f1eeeba48f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57"
I20260923 20:47:07.666280 13330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b9d8b9eb7b141c29d9395a3c5a055cd" candidate_uuid: "353283811d90497ca195f1eeeba48f99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e668f71bc0422cb74da107e401b435"
I20260923 20:47:07.666445 13193 raft_consensus.cc:3037] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.666531 13330 raft_consensus.cc:3037] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.670017 13193 raft_consensus.cc:2454] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353283811d90497ca195f1eeeba48f99 in term 1.
I20260923 20:47:07.670192 13330 raft_consensus.cc:2454] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 353283811d90497ca195f1eeeba48f99 in term 1.
I20260923 20:47:07.670610 13402 leader_election.cc:302] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, 353283811d90497ca195f1eeeba48f99; no voters:
I20260923 20:47:07.671030 13550 raft_consensus.cc:2781] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:07.671556 13550 raft_consensus.cc:686] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 1 LEADER]: Becoming Leader. State: Replica: 353283811d90497ca195f1eeeba48f99, State: Running, Role: LEADER
I20260923 20:47:07.672077 13550 consensus_queue.cc:237] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.676317 13032 catalog_manager.cc:5495] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 353283811d90497ca195f1eeeba48f99 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "353283811d90497ca195f1eeeba48f99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } health_report { overall_health: HEALTHY } } }
Sep 23 20:47:07 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196425, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.190@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:07.746126 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.722809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45149 (local address 127.0.0.1:53690)
0923 20:47:07.727462 (+ 4653us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.727497 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.727701 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.728117 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.728125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.729141 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:07.729553 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.729568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.731163 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.731170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.732959 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.732977 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.733979 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.733987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.734193 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.735061 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:07.735101 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:07.738180 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:07.742602 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:07.742617 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:07.742626 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:07.743202 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:07.743626 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:07.743631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:07.743635 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:07.743841 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:07.744525 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:07.744534 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:07.744880 (+ 346us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.745136 (+ 256us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.745585 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4471,"mutex_wait_us":6,"thread_start_us":4282,"threads_started":1}
I20260923 20:47:07.746174 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.722798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.5.71.190:45149)
0923 20:47:07.722994 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.723005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.727629 (+ 4624us) server_negotiation.cc:407] Connection header received
0923 20:47:07.727859 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.727864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.727939 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.728048 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:07.729837 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.731005 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.733127 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.733761 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.733894 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.738443 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:07.738451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:07.738456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:07.738497 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:07.742439 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.743353 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.743358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.743362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.743514 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:07.743987 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:07.743993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:07.743997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:07.744395 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:07.744541 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.745269 (+ 728us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.745742 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":209,"server-negotiator.queue_time_us":55}
I20260923 20:47:07.752835 13032 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:53690:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 20:47:07.754133 13032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:07.761613 13169 tablet_service.cc:1467] Processing CreateTablet for tablet a8261540b37146d4926345a09db6713e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1260c3cc5384557a97522761c93cb0c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:47:07.762198 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8261540b37146d4926345a09db6713e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.764358 13449 tablet_service.cc:1467] Processing CreateTablet for tablet a8261540b37146d4926345a09db6713e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1260c3cc5384557a97522761c93cb0c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:47:07.764873 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8261540b37146d4926345a09db6713e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.769464 13544 tablet_bootstrap.cc:492] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: Bootstrap starting.
I20260923 20:47:07.770221 13545 tablet_bootstrap.cc:492] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: Bootstrap starting.
I20260923 20:47:07.770155 13310 tablet_service.cc:1467] Processing CreateTablet for tablet a8261540b37146d4926345a09db6713e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1260c3cc5384557a97522761c93cb0c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 20:47:07.770725 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8261540b37146d4926345a09db6713e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:07.771199 13544 tablet_bootstrap.cc:654] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.771917 13545 tablet_bootstrap.cc:654] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.782056 13546 tablet_bootstrap.cc:492] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: Bootstrap starting.
I20260923 20:47:07.783352 13544 tablet_bootstrap.cc:492] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: No bootstrap required, opened a new log
I20260923 20:47:07.783514 13544 ts_tablet_manager.cc:1397] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260923 20:47:07.783871 13545 tablet_bootstrap.cc:492] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: No bootstrap required, opened a new log
I20260923 20:47:07.784053 13545 ts_tablet_manager.cc:1397] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260923 20:47:07.784128 13546 tablet_bootstrap.cc:654] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:07.784294 13544 raft_consensus.cc:348] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.784538 13544 raft_consensus.cc:374] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.784608 13544 raft_consensus.cc:729] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bd0c233537425eba8e2bbcc426ce57, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.784847 13545 raft_consensus.cc:348] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.784878 13544 consensus_queue.cc:260] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.785089 13545 raft_consensus.cc:374] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.785171 13545 raft_consensus.cc:729] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353283811d90497ca195f1eeeba48f99, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.785434 13545 consensus_queue.cc:260] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.785312 13551 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.785701 13551 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:47:07.785991 13545 ts_tablet_manager.cc:1428] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260923 20:47:07.786492 13550 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.786785 13550 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: This TxnStatusTablet replica's current role is: FOLLOWER
W20260923 20:47:07.787881 13521 tablet.cc:2369] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:47:07.788493 13544 ts_tablet_manager.cc:1428] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260923 20:47:07.793402 13546 tablet_bootstrap.cc:492] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: No bootstrap required, opened a new log
I20260923 20:47:07.793658 13546 ts_tablet_manager.cc:1397] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260923 20:47:07.794510 13546 raft_consensus.cc:348] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.794763 13546 raft_consensus.cc:374] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:07.794874 13546 raft_consensus.cc:729] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Initialized, Role: FOLLOWER
I20260923 20:47:07.795118 13546 consensus_queue.cc:260] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.795715 13546 ts_tablet_manager.cc:1428] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260923 20:47:07.795544 13563 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.795923 13563 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:47:07.840021 13563 raft_consensus.cc:482] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:07.840287 13563 raft_consensus.cc:504] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.841183 13563 leader_election.cc:288] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:07.841684 13193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8261540b37146d4926345a09db6713e" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57" is_pre_election: true
I20260923 20:47:07.841741 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8261540b37146d4926345a09db6713e" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99" is_pre_election: true
I20260923 20:47:07.841964 13193 raft_consensus.cc:2454] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 0.
I20260923 20:47:07.841997 13475 raft_consensus.cc:2454] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 0.
I20260923 20:47:07.842423 13262 leader_election.cc:302] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:07.842671 13563 raft_consensus.cc:2781] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:07.842793 13563 raft_consensus.cc:482] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:07.842896 13563 raft_consensus.cc:3037] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.845919 13563 raft_consensus.cc:504] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.846526 13563 leader_election.cc:288] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 election: Requested vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:07.847004 13193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8261540b37146d4926345a09db6713e" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57"
I20260923 20:47:07.847092 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8261540b37146d4926345a09db6713e" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99"
I20260923 20:47:07.847276 13193 raft_consensus.cc:3037] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.847316 13475 raft_consensus.cc:3037] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:07.850320 13193 raft_consensus.cc:2454] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 1.
I20260923 20:47:07.850338 13475 raft_consensus.cc:2454] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 1.
I20260923 20:47:07.850761 13263 leader_election.cc:302] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 353283811d90497ca195f1eeeba48f99, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:07.851118 13563 raft_consensus.cc:2781] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:07.851291 13563 raft_consensus.cc:686] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [term 1 LEADER]: Becoming Leader. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Running, Role: LEADER
I20260923 20:47:07.851591 13563 consensus_queue.cc:237] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:07.853497 13555 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: TxnStatusTablet state changed. Reason: New leader c2e668f71bc0422cb74da107e401b435. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.853922 13555 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:47:07.854662 13587 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 20:47:07.854892 13587 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 20:47:07.855654 13032 catalog_manager.cc:5495] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2e668f71bc0422cb74da107e401b435 (127.5.71.130). New cstate: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260923 20:47:07.873868 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.868211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49564 (local address 127.5.71.130:38499)
0923 20:47:07.868450 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.868457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.868575 (+ 118us) server_negotiation.cc:407] Connection header received
0923 20:47:07.868942 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.868948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.869074 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.869212 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:07.869598 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.870458 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.871949 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.872433 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.872593 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.873062 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.873172 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.873289 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.873455 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.873540 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260923 20:47:07.873915 13582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.868209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38499 (local address 127.0.0.1:49564)
0923 20:47:07.868478 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.868501 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.868742 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.869241 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.869247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.869259 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:47:07.869462 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.869471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.870610 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.870619 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.871749 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.871760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.872609 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.872614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.872811 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.872869 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.873287 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.873312 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.873421 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.873580 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260923 20:47:07.877840 13475 raft_consensus.cc:1261] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Refusing update from remote peer c2e668f71bc0422cb74da107e401b435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:07.877854 13193 raft_consensus.cc:1261] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Refusing update from remote peer c2e668f71bc0422cb74da107e401b435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:07.878469 13563 consensus_queue.cc:1035] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:07.878839 13555 consensus_queue.cc:1035] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:07.882640 13550 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: TxnStatusTablet state changed. Reason: New leader c2e668f71bc0422cb74da107e401b435. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.882979 13550 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:47:07.885591 13555 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.885870 13555 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:47:07.887673 13550 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.887980 13550 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P 353283811d90497ca195f1eeeba48f99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:47:07.888227 13560 mvcc.cc:205] Tried to move back new op lower bound from 7332644568580825088 to 7332644568483041280. Current Snapshot: MvccSnapshot[applied={T|T < 7332644568580825088 or (T in {7332644568580825088})}]
I20260923 20:47:07.889386 13551 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: TxnStatusTablet state changed. Reason: New leader c2e668f71bc0422cb74da107e401b435. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.889751 13551 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:47:07.891649 13552 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.892046 13552 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P c2e668f71bc0422cb74da107e401b435: This TxnStatusTablet replica's current role is: LEADER
I20260923 20:47:07.894945 13551 tablet_replica.cc:439] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } } }
I20260923 20:47:07.895198 13551 tablet_replica.cc:442] T a8261540b37146d4926345a09db6713e P 08bd0c233537425eba8e2bbcc426ce57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 20:47:07.902874 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.896961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.5.71.131:38397)
0923 20:47:07.897122 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.897129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.897147 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:47:07.897239 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.897245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.897389 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.897565 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:07.898055 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.899051 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.900553 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.901016 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.901163 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.901638 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.902234 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.902404 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.902466 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.902551 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 20:47:07.921731 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.914601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:52689 (local address 127.5.71.130:38499)
0923 20:47:07.914876 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.914887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.915360 (+ 473us) server_negotiation.cc:407] Connection header received
0923 20:47:07.915604 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.915608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.915699 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.915852 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:07.916416 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.917493 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.919497 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.920083 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.920216 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.920807 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.921033 (+ 226us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.921121 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.921321 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.921394 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:47:07.921849 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.914543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38499 (local address 127.5.71.131:52689)
0923 20:47:07.915242 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.915269 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.915418 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.915861 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.915866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.915881 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:47:07.916219 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.916231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.917633 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.917639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.919330 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.919346 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.920298 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.920305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.920503 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.920584 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.921146 (+ 562us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.921158 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.921251 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.921400 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":367,"threads_started":1}
I20260923 20:47:07.933068 13193 raft_consensus.cc:1261] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Refusing update from remote peer 353283811d90497ca195f1eeeba48f99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:07.933308 13330 raft_consensus.cc:1261] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Refusing update from remote peer 353283811d90497ca195f1eeeba48f99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:07.933960 13575 consensus_queue.cc:1035] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:07.934686 13550 consensus_queue.cc:1035] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:07.971688 13541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.964231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:40137 (local address 127.5.71.131:38397)
0923 20:47:07.964419 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:07.964428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:07.965205 (+ 777us) server_negotiation.cc:407] Connection header received
0923 20:47:07.965527 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:07.965533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:07.965631 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:07.965740 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:07.966456 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.967507 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.969633 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.970306 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.970457 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.970942 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.971088 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.971194 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:07.971393 (+ 199us) server_negotiation.cc:299] Negotiation successful
0923 20:47:07.971461 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:47:07.971930 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:07.964269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38397 (local address 127.5.71.130:40137)
0923 20:47:07.965108 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:07.965133 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:07.965323 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:07.965798 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:07.965803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:07.965816 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:47:07.966273 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.966284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.967643 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.967651 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.969478 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:07.969491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:07.970488 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:07.970494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:07.970668 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:07.970736 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:47:07.971212 (+ 476us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:47:07.971225 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 20:47:07.971322 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 20:47:07.971486 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":453,"threads_started":1}
I20260923 20:47:08.015426 13031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:08.015697 13031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:08.019724 13031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Sending DeleteTablet for 3 replicas of tablet 7b9d8b9eb7b141c29d9395a3c5a055cd
I20260923 20:47:08.020831 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57306
I20260923 20:47:08.021145 13569 tablet_replica.cc:330] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: stopping tablet replica
I20260923 20:47:08.021378 13569 raft_consensus.cc:2229] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:08.021356 13310 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:49548
I20260923 20:47:08.021605 13569 raft_consensus.cc:2258] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.021642 13568 tablet_replica.cc:330] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: stopping tablet replica
I20260923 20:47:08.021878 13568 raft_consensus.cc:2229] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:08.021919 13449 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b9d8b9eb7b141c29d9395a3c5a055cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:40330
I20260923 20:47:08.022151 13568 raft_consensus.cc:2258] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.022243 13570 tablet_replica.cc:330] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: stopping tablet replica
I20260923 20:47:08.022471 13570 raft_consensus.cc:2229] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:08.022989 13570 raft_consensus.cc:2258] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.023523 13569 ts_tablet_manager.cc:1905] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.028656 13570 ts_tablet_manager.cc:1905] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.037067 13568 ts_tablet_manager.cc:1905] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.045111 13570 ts_tablet_manager.cc:1918] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:47:08.047362 13569 ts_tablet_manager.cc:1918] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:47:08.047536 13569 log.cc:1196] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/7b9d8b9eb7b141c29d9395a3c5a055cd
I20260923 20:47:08.047744 13570 log.cc:1196] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/7b9d8b9eb7b141c29d9395a3c5a055cd
I20260923 20:47:08.048373 13570 ts_tablet_manager.cc:1939] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 353283811d90497ca195f1eeeba48f99: Deleting consensus metadata
I20260923 20:47:08.049777 13020 catalog_manager.cc:4841] TS 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397): tablet 7b9d8b9eb7b141c29d9395a3c5a055cd (table test-table [id=d50d0e37651c4cabaf0118570cf92ee5]) successfully deleted
I20260923 20:47:08.049899 13568 ts_tablet_manager.cc:1918] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:47:08.050081 13568 log.cc:1196] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/7b9d8b9eb7b141c29d9395a3c5a055cd
I20260923 20:47:08.050588 13568 ts_tablet_manager.cc:1939] T 7b9d8b9eb7b141c29d9395a3c5a055cd P c2e668f71bc0422cb74da107e401b435: Deleting consensus metadata
I20260923 20:47:08.049167 13569 ts_tablet_manager.cc:1939] T 7b9d8b9eb7b141c29d9395a3c5a055cd P 08bd0c233537425eba8e2bbcc426ce57: Deleting consensus metadata
I20260923 20:47:08.052495 13019 catalog_manager.cc:4841] TS 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837): tablet 7b9d8b9eb7b141c29d9395a3c5a055cd (table test-table [id=d50d0e37651c4cabaf0118570cf92ee5]) successfully deleted
I20260923 20:47:08.053462 13020 catalog_manager.cc:4841] TS c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499): tablet 7b9d8b9eb7b141c29d9395a3c5a055cd (table test-table [id=d50d0e37651c4cabaf0118570cf92ee5]) successfully deleted
Sep 23 20:47:08 dist-test-slave-dn29 krb5kdc[12988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 20:47:08 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:08 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 20:47:08.113169 13407 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:08.099731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.5.71.190:45149)
0923 20:47:08.099964 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.099971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.100516 (+ 545us) server_negotiation.cc:407] Connection header received
0923 20:47:08.100691 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.100698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.100800 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.100930 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:08.102535 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.103970 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.105806 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.106437 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.106570 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.112897 (+ 6327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53704: BlockingRecv error: recv got EOF from 127.0.0.1:53704 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Sep 23 20:47:08 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM
I20260923 20:47:08.135126 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.116142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.5.71.190:45149)
0923 20:47:08.116296 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.116302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.116803 (+ 501us) server_negotiation.cc:407] Connection header received
0923 20:47:08.116971 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.116976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.117071 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.117173 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:08.118861 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.119970 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.121978 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.122653 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.122754 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.128024 (+ 5270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:08.128031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:08.128037 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:08.128089 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:08.131643 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:08.132544 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:08.132550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:08.132553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:08.132709 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:08.133202 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:08.133208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:08.133213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:08.133553 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:08.133796 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.134478 (+ 682us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.134727 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 20:47:08.140129 13031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:08.140919 13031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:08.150257 13169 tablet_service.cc:1467] Processing CreateTablet for tablet 20ea2013a7594477a552a6396353d6e7 (DEFAULT_TABLE table=test-table [id=83b23871fb4a427a88c69f9965d6f375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:08.150766 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ea2013a7594477a552a6396353d6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:08.152177 13449 tablet_service.cc:1467] Processing CreateTablet for tablet 20ea2013a7594477a552a6396353d6e7 (DEFAULT_TABLE table=test-table [id=83b23871fb4a427a88c69f9965d6f375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:08.152638 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ea2013a7594477a552a6396353d6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:08.156873 13310 tablet_service.cc:1467] Processing CreateTablet for tablet 20ea2013a7594477a552a6396353d6e7 (DEFAULT_TABLE table=test-table [id=83b23871fb4a427a88c69f9965d6f375]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:08.157385 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ea2013a7594477a552a6396353d6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:08.165352 13546 tablet_bootstrap.cc:492] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Bootstrap starting.
I20260923 20:47:08.167227 13546 tablet_bootstrap.cc:654] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:08.175966 13545 tablet_bootstrap.cc:492] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Bootstrap starting.
I20260923 20:47:08.178010 13545 tablet_bootstrap.cc:654] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:08.182950 13544 tablet_bootstrap.cc:492] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Bootstrap starting.
I20260923 20:47:08.185032 13544 tablet_bootstrap.cc:654] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:08.189061 13544 tablet_bootstrap.cc:492] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: No bootstrap required, opened a new log
I20260923 20:47:08.189329 13544 ts_tablet_manager.cc:1397] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260923 20:47:08.190263 13544 raft_consensus.cc:348] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.190699 13544 raft_consensus.cc:374] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:08.190857 13544 raft_consensus.cc:729] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bd0c233537425eba8e2bbcc426ce57, State: Initialized, Role: FOLLOWER
I20260923 20:47:08.191212 13544 consensus_queue.cc:260] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.194932 13544 ts_tablet_manager.cc:1428] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260923 20:47:08.205790 13545 tablet_bootstrap.cc:492] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: No bootstrap required, opened a new log
I20260923 20:47:08.206077 13545 ts_tablet_manager.cc:1397] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.004s
I20260923 20:47:08.207098 13545 raft_consensus.cc:348] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.207512 13545 raft_consensus.cc:374] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:08.207667 13545 raft_consensus.cc:729] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353283811d90497ca195f1eeeba48f99, State: Initialized, Role: FOLLOWER
I20260923 20:47:08.208053 13545 consensus_queue.cc:260] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.208802 13545 ts_tablet_manager.cc:1428] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260923 20:47:08.214488 13546 tablet_bootstrap.cc:492] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: No bootstrap required, opened a new log
I20260923 20:47:08.214670 13546 ts_tablet_manager.cc:1397] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Time spent bootstrapping tablet: real 0.049s user 0.011s sys 0.024s
I20260923 20:47:08.215580 13546 raft_consensus.cc:348] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.215955 13546 raft_consensus.cc:374] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:08.216064 13546 raft_consensus.cc:729] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Initialized, Role: FOLLOWER
I20260923 20:47:08.216396 13546 consensus_queue.cc:260] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.217108 13546 ts_tablet_manager.cc:1428] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 20:47:08.305830 13555 raft_consensus.cc:482] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:08.306083 13555 raft_consensus.cc:504] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.306855 13555 leader_election.cc:288] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:08.307715 13193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea2013a7594477a552a6396353d6e7" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57" is_pre_election: true
I20260923 20:47:08.308017 13193 raft_consensus.cc:2454] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 0.
I20260923 20:47:08.308876 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea2013a7594477a552a6396353d6e7" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99" is_pre_election: true
I20260923 20:47:08.309144 13475 raft_consensus.cc:2454] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 0.
I20260923 20:47:08.309912 13262 leader_election.cc:302] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:08.310709 13555 raft_consensus.cc:2781] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:08.310847 13555 raft_consensus.cc:482] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:08.310941 13555 raft_consensus.cc:3037] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:08.314841 13555 raft_consensus.cc:504] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.316077 13193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea2013a7594477a552a6396353d6e7" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08bd0c233537425eba8e2bbcc426ce57"
I20260923 20:47:08.316308 13193 raft_consensus.cc:3037] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:08.319561 13193 raft_consensus.cc:2454] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 1.
I20260923 20:47:08.320199 13262 leader_election.cc:302] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:08.320542 13563 raft_consensus.cc:2781] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:08.320679 13563 raft_consensus.cc:686] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 1 LEADER]: Becoming Leader. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Running, Role: LEADER
I20260923 20:47:08.321012 13563 consensus_queue.cc:237] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.322963 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20ea2013a7594477a552a6396353d6e7" candidate_uuid: "c2e668f71bc0422cb74da107e401b435" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99"
I20260923 20:47:08.323190 13475 raft_consensus.cc:3037] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:08.326484 13475 raft_consensus.cc:2454] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2e668f71bc0422cb74da107e401b435 in term 1.
I20260923 20:47:08.327907 13555 leader_election.cc:288] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [CANDIDATE]: Term 1 election: Requested vote from peers 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:08.336907 13032 catalog_manager.cc:5495] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 reported cstate change: term changed from 0 to 1, leader changed from <none> to c2e668f71bc0422cb74da107e401b435 (127.5.71.130). New cstate: current_term: 1 leader_uuid: "c2e668f71bc0422cb74da107e401b435" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260923 20:47:08.377308 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.371747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.5.71.130:38499)
0923 20:47:08.372005 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.372011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.372037 (+ 26us) server_negotiation.cc:407] Connection header received
0923 20:47:08.372100 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.372103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.372170 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.372295 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:08.372677 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.373552 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.375266 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.375766 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.375982 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.376481 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:08.376616 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:08.376737 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.376905 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.376993 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260923 20:47:08.380419 13193 raft_consensus.cc:1261] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Refusing update from remote peer c2e668f71bc0422cb74da107e401b435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:08.380445 13475 raft_consensus.cc:1261] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Refusing update from remote peer c2e668f71bc0422cb74da107e401b435: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:08.381178 13555 consensus_queue.cc:1035] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:08.381814 13552 consensus_queue.cc:1035] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:08.407756 13032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:08.408074 13032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:08.411968 13032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Sending DeleteTablet for 3 replicas of tablet 20ea2013a7594477a552a6396353d6e7
I20260923 20:47:08.413012 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ea2013a7594477a552a6396353d6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57306
I20260923 20:47:08.413394 13569 tablet_replica.cc:330] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: stopping tablet replica
I20260923 20:47:08.413609 13569 raft_consensus.cc:2229] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:08.413887 13569 raft_consensus.cc:2258] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.414114 13310 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ea2013a7594477a552a6396353d6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:49548
I20260923 20:47:08.414472 13568 tablet_replica.cc:330] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: stopping tablet replica
I20260923 20:47:08.414485 13449 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ea2013a7594477a552a6396353d6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:40330
I20260923 20:47:08.414678 13568 raft_consensus.cc:2229] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:08.414776 13450 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ea2013a7594477a552a6396353d6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:40330
I20260923 20:47:08.415066 13568 raft_consensus.cc:2258] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.415617 13570 tablet_replica.cc:330] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: stopping tablet replica
I20260923 20:47:08.415853 13570 raft_consensus.cc:2229] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:08.416138 13570 raft_consensus.cc:2258] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.416234 13569 ts_tablet_manager.cc:1905] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.417178 13032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:08.418015 13032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:08.425010 13569 ts_tablet_manager.cc:1918] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:08.425428 13569 log.cc:1196] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/20ea2013a7594477a552a6396353d6e7
I20260923 20:47:08.426201 13569 ts_tablet_manager.cc:1939] T 20ea2013a7594477a552a6396353d6e7 P 08bd0c233537425eba8e2bbcc426ce57: Deleting consensus metadata
I20260923 20:47:08.426976 13169 tablet_service.cc:1467] Processing CreateTablet for tablet a9587dc106754c6886e2cbd6607071b3 (DEFAULT_TABLE table=test-table [id=0ee85269cab54e09bcf25f5b661038fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:08.427392 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9587dc106754c6886e2cbd6607071b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:08.432011 13544 tablet_bootstrap.cc:492] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Bootstrap starting.
I20260923 20:47:08.432703 13019 catalog_manager.cc:4841] TS 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837): tablet 20ea2013a7594477a552a6396353d6e7 (table test-table [id=83b23871fb4a427a88c69f9965d6f375]) successfully deleted
I20260923 20:47:08.433672 13544 tablet_bootstrap.cc:654] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:08.433568 13450 tablet_service.cc:1467] Processing CreateTablet for tablet a9587dc106754c6886e2cbd6607071b3 (DEFAULT_TABLE table=test-table [id=0ee85269cab54e09bcf25f5b661038fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:08.434024 13450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9587dc106754c6886e2cbd6607071b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:08.434958 13310 tablet_service.cc:1467] Processing CreateTablet for tablet a9587dc106754c6886e2cbd6607071b3 (DEFAULT_TABLE table=test-table [id=0ee85269cab54e09bcf25f5b661038fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:08.435407 13310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9587dc106754c6886e2cbd6607071b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:08.444195 13568 ts_tablet_manager.cc:1905] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.450850 13545 tablet_bootstrap.cc:492] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Bootstrap starting.
I20260923 20:47:08.450914 13568 ts_tablet_manager.cc:1918] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:08.451079 13568 log.cc:1196] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/20ea2013a7594477a552a6396353d6e7
I20260923 20:47:08.451399 13546 tablet_bootstrap.cc:492] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Bootstrap starting.
I20260923 20:47:08.452925 13544 tablet_bootstrap.cc:492] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: No bootstrap required, opened a new log
I20260923 20:47:08.452939 13545 tablet_bootstrap.cc:654] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:08.453197 13544 ts_tablet_manager.cc:1397] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.008s
I20260923 20:47:08.454169 13544 raft_consensus.cc:348] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.454543 13544 raft_consensus.cc:374] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:08.454694 13544 raft_consensus.cc:729] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bd0c233537425eba8e2bbcc426ce57, State: Initialized, Role: FOLLOWER
I20260923 20:47:08.455157 13544 consensus_queue.cc:260] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.457091 13544 ts_tablet_manager.cc:1428] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260923 20:47:08.459828 13570 ts_tablet_manager.cc:1905] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.463219 13545 tablet_bootstrap.cc:492] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: No bootstrap required, opened a new log
I20260923 20:47:08.463392 13545 ts_tablet_manager.cc:1397] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260923 20:47:08.463979 13546 tablet_bootstrap.cc:654] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:08.464350 13545 raft_consensus.cc:348] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.464640 13545 raft_consensus.cc:374] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:08.464723 13545 raft_consensus.cc:729] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353283811d90497ca195f1eeeba48f99, State: Initialized, Role: FOLLOWER
I20260923 20:47:08.464980 13545 consensus_queue.cc:260] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.466169 13545 ts_tablet_manager.cc:1428] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 20:47:08.467316 13568 ts_tablet_manager.cc:1939] T 20ea2013a7594477a552a6396353d6e7 P c2e668f71bc0422cb74da107e401b435: Deleting consensus metadata
I20260923 20:47:08.469570 13020 catalog_manager.cc:4841] TS c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499): tablet 20ea2013a7594477a552a6396353d6e7 (table test-table [id=83b23871fb4a427a88c69f9965d6f375]) successfully deleted
I20260923 20:47:08.470700 13570 ts_tablet_manager.cc:1918] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:08.470861 13570 log.cc:1196] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/20ea2013a7594477a552a6396353d6e7
I20260923 20:47:08.471289 13570 ts_tablet_manager.cc:1939] T 20ea2013a7594477a552a6396353d6e7 P 353283811d90497ca195f1eeeba48f99: Deleting consensus metadata
I20260923 20:47:08.472522 13020 catalog_manager.cc:4841] TS 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397): tablet 20ea2013a7594477a552a6396353d6e7 (table test-table [id=83b23871fb4a427a88c69f9965d6f375]) successfully deleted
W20260923 20:47:08.473150 13020 catalog_manager.cc:4808] TS 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397): delete failed for tablet 20ea2013a7594477a552a6396353d6e7 because the tablet was not found. No further retry: Not found: Tablet not found: 20ea2013a7594477a552a6396353d6e7
I20260923 20:47:08.474062 13546 tablet_bootstrap.cc:492] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: No bootstrap required, opened a new log
I20260923 20:47:08.474248 13546 ts_tablet_manager.cc:1397] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.000s
I20260923 20:47:08.475322 13546 raft_consensus.cc:348] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.475943 13546 raft_consensus.cc:374] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:08.476055 13546 raft_consensus.cc:729] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e668f71bc0422cb74da107e401b435, State: Initialized, Role: FOLLOWER
I20260923 20:47:08.476364 13546 consensus_queue.cc:260] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.477006 13546 ts_tablet_manager.cc:1428] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 20:47:08.489612 13551 raft_consensus.cc:482] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:08.489856 13551 raft_consensus.cc:504] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.491812 13551 leader_election.cc:288] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:08.500909 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.492354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:39737 (local address 127.5.71.130:38499)
0923 20:47:08.492564 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.492571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.492943 (+ 372us) server_negotiation.cc:407] Connection header received
0923 20:47:08.493239 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.493244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.493322 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.493394 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:08.494321 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.495443 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.498183 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.499488 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.499651 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.499741 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.500399 (+ 658us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.500558 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 20:47:08.501981 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.492067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38499 (local address 127.5.71.129:39737)
0923 20:47:08.492822 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:08.492847 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:08.493060 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:08.493625 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:08.493630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:08.493753 (+ 123us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:08.494097 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.494112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.495735 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.495741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.497956 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.497976 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.499969 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.499978 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.500183 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.500225 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 20:47:08.501410 (+ 1185us) client_negotiation.cc:240] Negotiation successful
0923 20:47:08.501587 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":510,"threads_started":1}
I20260923 20:47:08.504894 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9587dc106754c6886e2cbd6607071b3" candidate_uuid: "08bd0c233537425eba8e2bbcc426ce57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e668f71bc0422cb74da107e401b435" is_pre_election: true
I20260923 20:47:08.505169 13329 raft_consensus.cc:2454] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08bd0c233537425eba8e2bbcc426ce57 in term 0.
I20260923 20:47:08.505700 13123 leader_election.cc:302] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:08.506072 13551 raft_consensus.cc:2781] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:08.506172 13551 raft_consensus.cc:482] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:08.506251 13551 raft_consensus.cc:3037] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:08.509502 13551 raft_consensus.cc:504] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.510299 13551 leader_election.cc:288] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [CANDIDATE]: Term 1 election: Requested vote from peers c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499), 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397)
I20260923 20:47:08.511296 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.493021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38397 (local address 127.5.71.129:36413)
0923 20:47:08.502461 (+ 9440us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:08.502483 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:08.502862 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:08.504247 (+ 1385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:08.504252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:08.504282 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:08.504443 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.504453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.506406 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.506412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.508465 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.508483 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.510516 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.510522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.510725 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.510763 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 20:47:08.510851 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 20:47:08.511003 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9331,"thread_start_us":11222,"threads_started":1}
I20260923 20:47:08.511310 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9587dc106754c6886e2cbd6607071b3" candidate_uuid: "08bd0c233537425eba8e2bbcc426ce57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2e668f71bc0422cb74da107e401b435"
I20260923 20:47:08.511579 13329 raft_consensus.cc:3037] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:08.512100 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.493367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:36413 (local address 127.5.71.131:38397)
0923 20:47:08.503388 (+ 10021us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.503396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.503416 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:47:08.503489 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.503494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.503588 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.503697 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:08.504627 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.506218 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.508681 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.510086 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.510318 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.510407 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.511535 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.511712 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9888,"thread_start_us":9785,"threads_started":1}
I20260923 20:47:08.517287 13329 raft_consensus.cc:2454] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08bd0c233537425eba8e2bbcc426ce57 in term 1.
I20260923 20:47:08.518096 13123 leader_election.cc:302] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bd0c233537425eba8e2bbcc426ce57, c2e668f71bc0422cb74da107e401b435; no voters:
I20260923 20:47:08.518466 13551 raft_consensus.cc:2781] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:08.519295 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9587dc106754c6886e2cbd6607071b3" candidate_uuid: "08bd0c233537425eba8e2bbcc426ce57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99" is_pre_election: true
I20260923 20:47:08.519560 13475 raft_consensus.cc:2454] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08bd0c233537425eba8e2bbcc426ce57 in term 0.
I20260923 20:47:08.520290 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9587dc106754c6886e2cbd6607071b3" candidate_uuid: "08bd0c233537425eba8e2bbcc426ce57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "353283811d90497ca195f1eeeba48f99"
I20260923 20:47:08.520485 13475 raft_consensus.cc:3037] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:08.523715 13475 raft_consensus.cc:2454] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08bd0c233537425eba8e2bbcc426ce57 in term 1.
I20260923 20:47:08.524971 13551 raft_consensus.cc:686] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 LEADER]: Becoming Leader. State: Replica: 08bd0c233537425eba8e2bbcc426ce57, State: Running, Role: LEADER
I20260923 20:47:08.525470 13551 consensus_queue.cc:237] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } }
I20260923 20:47:08.530822 13032 catalog_manager.cc:5495] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "08bd0c233537425eba8e2bbcc426ce57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08bd0c233537425eba8e2bbcc426ce57" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 37837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 } health_report { overall_health: UNKNOWN } } }
I20260923 20:47:08.571362 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.561988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.5.71.129:37837)
0923 20:47:08.564241 (+ 2253us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.564248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.564270 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:47:08.564370 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.564375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.564474 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.564661 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:08.565122 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.566280 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.568281 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.569021 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.569163 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.569744 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:08.570571 (+ 827us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:08.570738 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.570873 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.570965 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2117,"thread_start_us":1987,"threads_started":1}
I20260923 20:47:08.594967 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.584571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:50905 (local address 127.5.71.130:38499)
0923 20:47:08.584752 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.584759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.587831 (+ 3072us) server_negotiation.cc:407] Connection header received
0923 20:47:08.588069 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.588075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.588158 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.588267 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:08.588905 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.590103 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.592334 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.593166 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.593297 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.594013 (+ 716us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:08.594200 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:08.594308 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.594482 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.594610 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:47:08.595094 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.584420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38499 (local address 127.5.71.129:50905)
0923 20:47:08.587668 (+ 3248us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:08.587689 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:08.587909 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:08.588316 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:08.588321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:08.588334 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:47:08.588706 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.588721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.590268 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.590275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.592142 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.592163 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.593462 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.593468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.593655 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.593791 (+ 136us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:47:08.594321 (+ 530us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:47:08.594333 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 20:47:08.594452 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 20:47:08.594653 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3036,"thread_start_us":2836,"threads_started":1}
I20260923 20:47:08.607556 13475 raft_consensus.cc:1261] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Refusing update from remote peer 08bd0c233537425eba8e2bbcc426ce57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:08.607587 13329 raft_consensus.cc:1261] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Refusing update from remote peer 08bd0c233537425eba8e2bbcc426ce57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:08.608337 13625 consensus_queue.cc:1035] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "353283811d90497ca195f1eeeba48f99" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:08.609367 13625 consensus_queue.cc:1035] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2e668f71bc0422cb74da107e401b435" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:08.654875 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.643762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33099 (local address 127.5.71.129:37837)
0923 20:47:08.643948 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.643953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.648395 (+ 4442us) server_negotiation.cc:407] Connection header received
0923 20:47:08.648624 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.648630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.648751 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.648837 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:08.649497 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.650561 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.652731 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.653160 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.653361 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.654011 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:08.654166 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:08.654259 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.654447 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.654525 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 20:47:08.654968 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.643270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:37837 (local address 127.5.71.130:33099)
0923 20:47:08.648286 (+ 5016us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:08.648303 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:08.648489 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:08.648865 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:08.648870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:08.648884 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 20:47:08.649348 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.649359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.650712 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.650717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.652560 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:08.652581 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.653587 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.653597 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:08.653799 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.653838 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 20:47:08.654311 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 20:47:08.654324 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 20:47:08.654425 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 20:47:08.654566 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4834,"thread_start_us":4700,"threads_started":1}
I20260923 20:47:08.692721 13031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:08.692998 13031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:08.696434 13031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a20db410821a42c58c37307c648d768b: Sending DeleteTablet for 3 replicas of tablet a9587dc106754c6886e2cbd6607071b3
I20260923 20:47:08.697436 13310 tablet_service.cc:1514] Processing DeleteTablet for tablet a9587dc106754c6886e2cbd6607071b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:49548
I20260923 20:47:08.697527 13450 tablet_service.cc:1514] Processing DeleteTablet for tablet a9587dc106754c6886e2cbd6607071b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:40330
I20260923 20:47:08.697785 13570 tablet_replica.cc:330] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: stopping tablet replica
I20260923 20:47:08.697808 13568 tablet_replica.cc:330] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: stopping tablet replica
I20260923 20:47:08.697988 13570 raft_consensus.cc:2229] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:08.697992 13568 raft_consensus.cc:2229] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:08.698187 13568 raft_consensus.cc:2258] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.698182 13570 raft_consensus.cc:2258] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.698728 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet a9587dc106754c6886e2cbd6607071b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:08 UTC) from {username='oryx', principal='oryx/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57306
I20260923 20:47:08.699033 13569 tablet_replica.cc:330] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: stopping tablet replica
I20260923 20:47:08.699249 13569 raft_consensus.cc:2229] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:08.699661 13568 ts_tablet_manager.cc:1905] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.699759 13569 raft_consensus.cc:2258] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:08.701391 13569 ts_tablet_manager.cc:1905] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:08.701561 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13104
I20260923 20:47:08.707808 13568 ts_tablet_manager.cc:1918] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:47:08.708002 13568 log.cc:1196] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/a9587dc106754c6886e2cbd6607071b3
I20260923 20:47:08.708031 13569 ts_tablet_manager.cc:1918] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:47:08.708186 13569 log.cc:1196] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/a9587dc106754c6886e2cbd6607071b3
I20260923 20:47:08.708534 13568 ts_tablet_manager.cc:1939] T a9587dc106754c6886e2cbd6607071b3 P c2e668f71bc0422cb74da107e401b435: Deleting consensus metadata
I20260923 20:47:08.708575 13569 ts_tablet_manager.cc:1939] T a9587dc106754c6886e2cbd6607071b3 P 08bd0c233537425eba8e2bbcc426ce57: Deleting consensus metadata
I20260923 20:47:08.709649 13019 catalog_manager.cc:4841] TS 08bd0c233537425eba8e2bbcc426ce57 (127.5.71.129:37837): tablet a9587dc106754c6886e2cbd6607071b3 (table test-table [id=0ee85269cab54e09bcf25f5b661038fa]) successfully deleted
I20260923 20:47:08.709905 13020 catalog_manager.cc:4841] TS c2e668f71bc0422cb74da107e401b435 (127.5.71.130:38499): tablet a9587dc106754c6886e2cbd6607071b3 (table test-table [id=0ee85269cab54e09bcf25f5b661038fa]) successfully deleted
I20260923 20:47:08.711958 13570 ts_tablet_manager.cc:1905] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 20:47:08 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.129@KRBTEST.COM
I20260923 20:47:08.728986 13570 ts_tablet_manager.cc:1918] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 20:47:08.729153 13570 log.cc:1196] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/a9587dc106754c6886e2cbd6607071b3
I20260923 20:47:08.729588 13570 ts_tablet_manager.cc:1939] T a9587dc106754c6886e2cbd6607071b3 P 353283811d90497ca195f1eeeba48f99: Deleting consensus metadata
I20260923 20:47:08.730815 13020 catalog_manager.cc:4841] TS 353283811d90497ca195f1eeeba48f99 (127.5.71.131:38397): tablet a9587dc106754c6886e2cbd6607071b3 (table test-table [id=0ee85269cab54e09bcf25f5b661038fa]) successfully deleted
I20260923 20:47:08.733031 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.704741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.5.71.129:37837)
0923 20:47:08.704937 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.704944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.714132 (+ 9188us) server_negotiation.cc:407] Connection header received
0923 20:47:08.714363 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.714370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.714457 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.714557 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:08.716318 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.717429 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.719353 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.719943 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.720028 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.725270 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:08.725286 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:08.725311 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:08.725353 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:08.729745 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:08.730604 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:08.730614 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:08.730618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:08.730775 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:08.731278 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:08.731284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:08.731287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:08.731765 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:08.732145 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.732375 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.732621 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:47:08.733799 13235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:08.841079 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13104
I20260923 20:47:08.860405 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13244
Sep 23 20:47:08 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.130@KRBTEST.COM
I20260923 20:47:08.875603 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:08.861220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.5.71.130:38499)
0923 20:47:08.861418 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:08.861426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:08.861513 (+ 87us) server_negotiation.cc:407] Connection header received
0923 20:47:08.861721 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:08.861727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:08.861798 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:08.861903 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:08.863197 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.863954 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.865203 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:08.865601 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:08.865733 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:08.869962 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:08.869976 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:08.869995 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:08.870029 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:08.872918 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:08.873588 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:08.873596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:08.873598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:08.873720 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:08.874069 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:08.874076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:08.874079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:08.874423 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:08.874748 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:08.875034 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 20:47:08.875271 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:47:08.876268 13375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:09.024684 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13244
I20260923 20:47:09.054251 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13384
Sep 23 20:47:09 dist-test-slave-dn29 krb5kdc[12988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.71.131@KRBTEST.COM
I20260923 20:47:09.071528 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:09.055217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.5.71.131:38397)
0923 20:47:09.055707 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:09.055711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:09.055727 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:47:09.055782 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:09.055833 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:09.055936 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:09.056074 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:09.057506 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:09.058294 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:09.059723 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:09.060322 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:09.060465 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:09.065214 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:09.065232 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:09.065253 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:09.065296 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:09.068349 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:09.069114 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:09.069129 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:09.069131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:09.069282 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:09.069723 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:09.069730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:09.069734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:09.070167 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:09.070482 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:09.070861 (+ 379us) server_negotiation.cc:299] Negotiation successful
0923 20:47:09.071120 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":240,"threads_started":1}
I20260923 20:47:09.072417 13515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:09.197556 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13384
I20260923 20:47:09.219350 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13001
I20260923 20:47:09.220808 13073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:09.341073 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13001
2026-09-23T20:47:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5648 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:47:09 dist-test-slave-dn29 krb5kdc[13646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:47:09 dist-test-slave-dn29 krb5kdc[13646](info): set up 2 sockets
Sep 23 20:47:09 dist-test-slave-dn29 krb5kdc[13646](info): commencing operation
krb5kdc: starting...
W20260923 20:47:10.485410 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:47:10 dist-test-slave-dn29 krb5kdc[13646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:47:10.732154 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:34279
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:34279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:11.062749 13659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:11.063102 13659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:11.063287 13659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:11.070591 13659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:11.070779 13659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:11.070895 13659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:11.071017 13659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:11.071106 13659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:11.081440 13659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:34279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:34279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:11.082890 13659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:11.084410 13659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:11.097790 13667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:11.097735 13664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:11.097851 13665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:11.104077 13659 server_base.cc:1034] running on GCE node
I20260923 20:47:11.104998 13659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:11.106534 13659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:11.107915 13659 hybrid_clock.cc:648] HybridClock initialized: now 1790196431107765 us; error 46 us; skew 500 ppm
I20260923 20:47:11.108364 13659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T20:47:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1839 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:47:11 dist-test-slave-dn29 krb5kdc[13673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:47:11 dist-test-slave-dn29 krb5kdc[13673](info): set up 2 sockets
Sep 23 20:47:11 dist-test-slave-dn29 krb5kdc[13673](info): commencing operation
krb5kdc: starting...
W20260923 20:47:12.325649 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:47:12 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:47:12.572289 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45569
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:12.909801 13686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:12.910116 13686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:12.910279 13686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:12.917687 13686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:12.917816 13686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:12.917867 13686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:12.917989 13686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:12.918097 13686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:12.928632 13686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:12.930089 13686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:12.931468 13686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:12.944089 13691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:12.944260 13694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:12.944260 13692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:12.945489 13686 server_base.cc:1034] running on GCE node
I20260923 20:47:12.946380 13686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:12.947993 13686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:12.949255 13686 hybrid_clock.cc:648] HybridClock initialized: now 1790196432949194 us; error 52 us; skew 500 ppm
I20260923 20:47:12.949581 13686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:12 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196432, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:12.961760 13686 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:47:12.963471 13686 webserver.cc:460] Webserver started at http://127.5.71.190:35301/ using document root <none> and password file <none>
I20260923 20:47:12.964316 13686 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:12.964421 13686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:12.964697 13686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:12.967905 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "bd91bb8101a14936927f21b59ede20c2"
format_stamp: "Formatted at 2026-09-23 20:47:12 on dist-test-slave-dn29"
server_key: "fc3af0d7fa03d1ade8b02278b61d527a"
server_key_iv: "95fe7bc5ec70e4e491c59654e6720b8b"
server_key_version: "encryptionkey@0"
I20260923 20:47:12.968664 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "bd91bb8101a14936927f21b59ede20c2"
format_stamp: "Formatted at 2026-09-23 20:47:12 on dist-test-slave-dn29"
server_key: "fc3af0d7fa03d1ade8b02278b61d527a"
server_key_iv: "95fe7bc5ec70e4e491c59654e6720b8b"
server_key_version: "encryptionkey@0"
I20260923 20:47:12.974285 13686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:47:12.978319 13701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:12.980176 13686 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:47:12.980373 13686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "bd91bb8101a14936927f21b59ede20c2"
format_stamp: "Formatted at 2026-09-23 20:47:12 on dist-test-slave-dn29"
server_key: "fc3af0d7fa03d1ade8b02278b61d527a"
server_key_iv: "95fe7bc5ec70e4e491c59654e6720b8b"
server_key_version: "encryptionkey@0"
I20260923 20:47:12.980574 13686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:12.999496 13686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:13.004554 13686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:13.004823 13686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:13.026165 13686 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45569
I20260923 20:47:13.026201 13752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45569 every 8 connection(s)
I20260923 20:47:13.027837 13686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:47:13.032152 13753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:13.035647 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13686
I20260923 20:47:13.035887 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:47:13.036724 5406 external_mini_cluster.cc:1428] Setting key d610dafdd029fb87c29a08529c377850
I20260923 20:47:13.045890 13753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Bootstrap starting.
I20260923 20:47:13.049234 13753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:13.050369 13753 log.cc:824] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:47:13 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:13.053962 13753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: No bootstrap required, opened a new log
I20260923 20:47:13.061544 13756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:13.039238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.5.71.190:45569)
0923 20:47:13.040204 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:13.040218 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:13.040297 (+ 79us) server_negotiation.cc:407] Connection header received
0923 20:47:13.042123 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:13.042151 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:13.042636 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:13.043147 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:13.044796 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:13.046197 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:13.047515 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:13.048340 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:13.048548 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:13.053428 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:13.053437 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:13.053457 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:13.053497 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:13.056993 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:13.057916 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:13.057923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:13.057934 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:13.058136 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:13.058571 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:13.058574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:13.058576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:13.059139 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:13.059385 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:13.059957 (+ 572us) server_negotiation.cc:299] Negotiation successful
0923 20:47:13.060409 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":514,"threads_started":1}
I20260923 20:47:13.063285 13753 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:13.063675 13753 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:13.063835 13753 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd91bb8101a14936927f21b59ede20c2, State: Initialized, Role: FOLLOWER
I20260923 20:47:13.064450 13753 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:13.064728 13753 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:47:13.064877 13753 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:47:13.065069 13753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:13.067341 13753 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:13.067984 13753 leader_election.cc:302] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd91bb8101a14936927f21b59ede20c2; no voters:
I20260923 20:47:13.068620 13753 leader_election.cc:288] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:47:13.068871 13758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:13.070152 13758 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 LEADER]: Becoming Leader. State: Replica: bd91bb8101a14936927f21b59ede20c2, State: Running, Role: LEADER
I20260923 20:47:13.070748 13758 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:13.071105 13753 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:47:13.073894 13760 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd91bb8101a14936927f21b59ede20c2. Latest consensus state: current_term: 1 leader_uuid: "bd91bb8101a14936927f21b59ede20c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } } }
I20260923 20:47:13.074460 13759 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd91bb8101a14936927f21b59ede20c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } } }
I20260923 20:47:13.074800 13759 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:13.074437 13760 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:13.078081 13767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:13.082455 13767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:13.091534 13767 catalog_manager.cc:1347] Generated new cluster ID: cc7de454cf954f708b72042a03a3f0a2
I20260923 20:47:13.091707 13767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:13.101907 13767 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:47:13.103662 13767 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:13.110611 13767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Generated new TSK 0
I20260923 20:47:13.111680 13767 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:47:13.180719 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45569
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:13.518368 13779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:13.518738 13779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:13.518911 13779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:13.526440 13779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:13.526633 13779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:13.526818 13779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:47:13.537163 13779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:13.538661 13779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:13.540144 13779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:13.554275 13787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:13.554317 13785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:13.554503 13784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:13.555063 13779 server_base.cc:1034] running on GCE node
I20260923 20:47:13.555680 13779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:13.556645 13779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:13.557894 13779 hybrid_clock.cc:648] HybridClock initialized: now 1790196433557842 us; error 42 us; skew 500 ppm
I20260923 20:47:13.558254 13779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:13 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:13.561828 13779 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:47:13.563387 13779 webserver.cc:460] Webserver started at http://127.5.71.129:40391/ using document root <none> and password file <none>
I20260923 20:47:13.564164 13779 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:13.564273 13779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:13.564523 13779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:13.567469 13779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "c1c3a739a8d148d3a493a72716ff3580"
format_stamp: "Formatted at 2026-09-23 20:47:13 on dist-test-slave-dn29"
server_key: "454f22053d3ccd9b6cc834626113ea99"
server_key_iv: "b7f033493460edc60bd07ba7019cabe5"
server_key_version: "encryptionkey@0"
I20260923 20:47:13.568274 13779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "c1c3a739a8d148d3a493a72716ff3580"
format_stamp: "Formatted at 2026-09-23 20:47:13 on dist-test-slave-dn29"
server_key: "454f22053d3ccd9b6cc834626113ea99"
server_key_iv: "b7f033493460edc60bd07ba7019cabe5"
server_key_version: "encryptionkey@0"
I20260923 20:47:13.573791 13779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:47:13.578078 13794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:13.579694 13779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:47:13.579988 13779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "c1c3a739a8d148d3a493a72716ff3580"
format_stamp: "Formatted at 2026-09-23 20:47:13 on dist-test-slave-dn29"
server_key: "454f22053d3ccd9b6cc834626113ea99"
server_key_iv: "b7f033493460edc60bd07ba7019cabe5"
server_key_version: "encryptionkey@0"
I20260923 20:47:13.580186 13779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:13.602854 13779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:13.607357 13779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:13.607676 13779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:13.608749 13779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:13.610358 13779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:13.610433 13779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:13.610553 13779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:13.610615 13779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:13.646468 13779 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:40927
I20260923 20:47:13.646522 13906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:40927 every 8 connection(s)
I20260923 20:47:13.647874 13779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:47:13.657054 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13779
I20260923 20:47:13.657282 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:47:13.657987 5406 external_mini_cluster.cc:1428] Setting key 6f65082f1716e7b146e21e484b39c0b3
Sep 23 20:47:13 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196433, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:13.668339 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:13.650501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:56533 (local address 127.5.71.190:45569)
0923 20:47:13.651074 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:13.651079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:13.651872 (+ 793us) server_negotiation.cc:407] Connection header received
0923 20:47:13.653816 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:13.653823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:13.653916 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:13.654046 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:13.655557 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:13.656344 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:13.657992 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:13.658450 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:13.658585 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:13.662639 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:13.662645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:13.662648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:13.662686 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:13.665273 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:13.666001 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:13.666008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:13.666011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:13.666140 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:13.666568 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:13.666573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:13.666577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:13.666841 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:13.666996 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:13.667719 (+ 723us) server_negotiation.cc:299] Negotiation successful
0923 20:47:13.667967 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":366,"threads_started":1}
I20260923 20:47:13.669338 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:13.650918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:56533)
0923 20:47:13.651672 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:13.651720 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:13.653538 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:13.654291 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:13.654299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:13.654804 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:13.655379 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:13.655396 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:13.656501 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:13.656505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:13.657851 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:13.657863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:13.658691 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:13.658700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:13.658933 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:13.659751 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:13.659775 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:13.662417 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:13.665420 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:13.665429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:13.665448 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:13.665880 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:13.666262 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:13.666267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:13.666271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:13.666456 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:13.666969 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:13.666977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:13.667342 (+ 365us) client_negotiation.cc:769] Sending connection context
0923 20:47:13.667617 (+ 275us) client_negotiation.cc:240] Negotiation successful
0923 20:47:13.668189 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":326,"threads_started":1}
I20260923 20:47:13.671054 13907 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45569
I20260923 20:47:13.671459 13907 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:13.672457 13907 heartbeater.cc:511] Master 127.5.71.190:45569 requested a full tablet report, sending...
I20260923 20:47:13.675127 13718 ts_manager.cc:194] Registered new tserver with Master: c1c3a739a8d148d3a493a72716ff3580 (127.5.71.129:40927)
I20260923 20:47:13.677636 13718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:56533
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:47:13.715957 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45569
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:14.048424 13913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:14.048732 13913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:14.048899 13913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:14.056453 13913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:14.056577 13913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:14.056762 13913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:47:14.067065 13913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:14.068666 13913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:14.069991 13913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:14.084563 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:14.084543 13921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:14.084782 13918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:14.086504 13913 server_base.cc:1034] running on GCE node
I20260923 20:47:14.087212 13913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:14.088501 13913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:14.089807 13913 hybrid_clock.cc:648] HybridClock initialized: now 1790196434089793 us; error 80 us; skew 500 ppm
I20260923 20:47:14.090219 13913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:14 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196434, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:14.094005 13913 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:47:14.095860 13913 webserver.cc:460] Webserver started at http://127.5.71.130:43177/ using document root <none> and password file <none>
I20260923 20:47:14.096678 13913 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:14.096853 13913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:14.097213 13913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:14.100790 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "c4de0dbde72b40fd8190c6425e4235e7"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1b34d625852b0727fcb4dc33dabfcfe1"
server_key_iv: "1f699fa757566a8acb864abf0f8c2f7f"
server_key_version: "encryptionkey@0"
I20260923 20:47:14.101698 13913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "c4de0dbde72b40fd8190c6425e4235e7"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1b34d625852b0727fcb4dc33dabfcfe1"
server_key_iv: "1f699fa757566a8acb864abf0f8c2f7f"
server_key_version: "encryptionkey@0"
I20260923 20:47:14.107980 13913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260923 20:47:14.112196 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:14.113678 13913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260923 20:47:14.113945 13913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "c4de0dbde72b40fd8190c6425e4235e7"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1b34d625852b0727fcb4dc33dabfcfe1"
server_key_iv: "1f699fa757566a8acb864abf0f8c2f7f"
server_key_version: "encryptionkey@0"
I20260923 20:47:14.114151 13913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:14.130257 13913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:14.135088 13913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:14.135401 13913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:14.136511 13913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:14.137929 13913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:14.138010 13913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:14.138144 13913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:14.138195 13913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:14.175019 13913 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:46221
I20260923 20:47:14.175071 14040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:46221 every 8 connection(s)
I20260923 20:47:14.176448 13913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:47:14.179680 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 13913
I20260923 20:47:14.179883 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:47:14.180516 5406 external_mini_cluster.cc:1428] Setting key 311efc0faf012d0dd69ef619f095e5cb
Sep 23 20:47:14 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196434, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:14.199069 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:14.179208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:36813 (local address 127.5.71.190:45569)
0923 20:47:14.179663 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:14.179668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:14.181172 (+ 1504us) server_negotiation.cc:407] Connection header received
0923 20:47:14.183306 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:14.183311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:14.183423 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:14.183543 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:14.185318 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.186229 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.187823 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.188252 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.188393 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:14.192508 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:14.192513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:14.192517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:14.192550 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:14.195425 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:14.196403 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:14.196416 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:14.196419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:14.196565 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:14.197077 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:14.197088 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:14.197090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:14.197401 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:14.197579 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:14.198406 (+ 827us) server_negotiation.cc:299] Negotiation successful
0923 20:47:14.198661 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":248,"threads_started":1}
I20260923 20:47:14.200307 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:14.179982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:36813)
0923 20:47:14.180963 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:14.181003 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:14.183014 (+ 2011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:14.183854 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:14.183863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:14.184428 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:14.185137 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:14.185157 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.186382 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.186391 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:14.187641 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:14.187660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.188489 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.188496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:14.188741 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:14.189524 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:14.189552 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:14.192260 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:14.195625 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:14.195637 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:14.195667 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:14.196246 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:14.196725 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:14.196730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:14.196732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:14.196940 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:14.197549 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:14.197559 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:14.197973 (+ 414us) client_negotiation.cc:769] Sending connection context
0923 20:47:14.198307 (+ 334us) client_negotiation.cc:240] Negotiation successful
0923 20:47:14.198953 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":425,"threads_started":1}
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:47:14.202246 14041 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45569
I20260923 20:47:14.202572 14041 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:14.203478 14041 heartbeater.cc:511] Master 127.5.71.190:45569 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:47:14.205368 13718 ts_manager.cc:194] Registered new tserver with Master: c4de0dbde72b40fd8190c6425e4235e7 (127.5.71.130:46221)
I20260923 20:47:14.206292 13718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:36813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:47:14.236729 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45569
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:14.579716 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:14.580044 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:14.580143 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:14.587883 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:14.587991 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:14.588105 14047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:14.598361 14047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:14.599653 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:14.601100 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:14.614827 14053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:14.614809 14052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:14.615010 14055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:14.615489 14047 server_base.cc:1034] running on GCE node
I20260923 20:47:14.616307 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:14.617539 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:14.618913 14047 hybrid_clock.cc:648] HybridClock initialized: now 1790196434618843 us; error 67 us; skew 500 ppm
I20260923 20:47:14.619272 14047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:14 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196434, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:14.622952 14047 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:47:14.624688 14047 webserver.cc:460] Webserver started at http://127.5.71.131:33209/ using document root <none> and password file <none>
I20260923 20:47:14.625440 14047 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:14.625587 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:14.625906 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:14.629069 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "e69d59e384d34bc4a8b60e79db50678f"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1e289d4f06b0026c785e1b0bcf7bb6e7"
server_key_iv: "cecc888a583099a4fecf370c629fc341"
server_key_version: "encryptionkey@0"
I20260923 20:47:14.629951 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "e69d59e384d34bc4a8b60e79db50678f"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1e289d4f06b0026c785e1b0bcf7bb6e7"
server_key_iv: "cecc888a583099a4fecf370c629fc341"
server_key_version: "encryptionkey@0"
I20260923 20:47:14.635555 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260923 20:47:14.639583 14062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:14.641263 14047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:47:14.641487 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "e69d59e384d34bc4a8b60e79db50678f"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1e289d4f06b0026c785e1b0bcf7bb6e7"
server_key_iv: "cecc888a583099a4fecf370c629fc341"
server_key_version: "encryptionkey@0"
I20260923 20:47:14.641808 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:14.675366 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:14.680739 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:14.681088 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:14.682127 13907 heartbeater.cc:503] Master 127.5.71.190:45569 was elected leader, sending a full tablet report...
I20260923 20:47:14.682281 14047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:14.684262 14047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:14.684383 14047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:14.684535 14047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:14.684617 14047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:14.723014 14047 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:43687
I20260923 20:47:14.723049 14174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:43687 every 8 connection(s)
I20260923 20:47:14.724565 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:14.731043 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14047
I20260923 20:47:14.731258 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:47:14.731935 5406 external_mini_cluster.cc:1428] Setting key 3402b7652c9a284652743121e5519ccd
Sep 23 20:47:14 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196434, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:14.748911 14177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:14.727436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:57839 (local address 127.5.71.190:45569)
0923 20:47:14.728006 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:14.728011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:14.729269 (+ 1258us) server_negotiation.cc:407] Connection header received
0923 20:47:14.732034 (+ 2765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:14.732043 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:14.732141 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:14.732248 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:14.734393 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.735957 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.737744 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.738193 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.738334 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:14.742260 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:14.742266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:14.742270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:14.742305 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:14.745171 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:14.746184 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:14.746190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:14.746193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:14.746384 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:14.747013 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:14.747021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:14.747026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:14.747341 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:14.747515 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:14.748305 (+ 790us) server_negotiation.cc:299] Negotiation successful
0923 20:47:14.748525 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":262,"threads_started":1}
I20260923 20:47:14.749961 14178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:14.728126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:57839)
0923 20:47:14.728885 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:14.729105 (+ 220us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:14.731621 (+ 2516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:14.732617 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:14.732627 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:14.733355 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:14.734206 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:14.734229 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.736166 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.736171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:14.737557 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:14.737576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.738447 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.738456 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:14.738714 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:14.739392 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:14.739417 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:14.742001 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:14.745351 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:14.745361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:14.745386 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:14.746006 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:14.746556 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:14.746561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:14.746564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:14.746916 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:14.747498 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:14.747506 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:14.747920 (+ 414us) client_negotiation.cc:769] Sending connection context
0923 20:47:14.748204 (+ 284us) client_negotiation.cc:240] Negotiation successful
0923 20:47:14.748751 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":323,"threads_started":1}
I20260923 20:47:14.751407 14175 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45569
I20260923 20:47:14.751848 14175 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:14.752589 14175 heartbeater.cc:511] Master 127.5.71.190:45569 requested a full tablet report, sending...
I20260923 20:47:14.754199 13718 ts_manager.cc:194] Registered new tserver with Master: e69d59e384d34bc4a8b60e79db50678f (127.5.71.131:43687)
I20260923 20:47:14.755090 13718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:57839
I20260923 20:47:14.760854 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab.
Entry for principal oryx/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab.
WARNING: no policy specified for oryx/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab.
Entry for principal oryx/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab.
WARNING: no policy specified for oryx/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab.
Entry for principal oryx/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab.
I20260923 20:47:14.911947 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13779
Sep 23 20:47:14 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:47:14.930740 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:14.913300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.5.71.129:40927)
0923 20:47:14.914019 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:14.914027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:14.914053 (+ 26us) server_negotiation.cc:407] Connection header received
0923 20:47:14.914125 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:14.914131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:14.914347 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:14.914498 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:14.916184 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.917227 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.918706 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:14.919397 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:14.919503 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:14.924037 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:14.924051 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:14.924069 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:14.924111 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:14.927497 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:14.928537 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:14.928549 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:14.928556 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:14.928794 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:14.929195 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:14.929198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:14.929200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:14.929554 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:14.929867 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:14.930135 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 20:47:14.930354 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":386,"threads_started":1}
I20260923 20:47:14.932124 13902 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:15.017410 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13779
I20260923 20:47:15.033833 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13913
Sep 23 20:47:15 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:47:15.052417 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:15.034893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.5.71.130:46221)
0923 20:47:15.035511 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:15.035520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:15.035548 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:47:15.035618 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:15.035645 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:15.036116 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:15.036318 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:15.037696 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:15.038816 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:15.040312 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:15.040956 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:15.041108 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:15.046038 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:15.046052 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:15.046072 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:15.046116 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:15.049471 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:15.050219 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:15.050227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:15.050229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:15.050405 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:15.050857 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:15.050861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:15.050864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:15.051213 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:15.051492 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:15.051762 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 20:47:15.052093 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":307,"threads_started":1}
I20260923 20:47:15.053774 14036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:15.144583 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13913
I20260923 20:47:15.160753 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14047
Sep 23 20:47:15 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:47:15.178773 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:15.161714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.5.71.131:43687)
0923 20:47:15.162244 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:15.162251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:15.162280 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:47:15.162357 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:15.162364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:15.162588 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:15.162714 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:15.164049 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:15.164997 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:15.166228 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:15.166864 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:15.166972 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:15.171939 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:15.171957 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:15.171977 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:15.172023 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:15.175528 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:15.176473 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:15.176481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:15.176484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:15.176632 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:15.177154 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:15.177157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:15.177160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:15.177520 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:15.177838 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:15.178069 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 20:47:15.178309 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
I20260923 20:47:15.180224 14170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:15.265018 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14047
I20260923 20:47:15.282660 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 13686
I20260923 20:47:15.283985 13748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:15.391259 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 13686
I20260923 20:47:15.412563 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45569
--webserver_interface=127.5.71.190
--webserver_port=35301
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:15.775977 14193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:15.776333 14193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:15.776505 14193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:15.784811 14193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:15.784933 14193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:15.785024 14193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:15.785116 14193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:15.785159 14193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:15.796026 14193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=35301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:15.797398 14193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:15.798744 14193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:15.811108 14201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:15.811182 14198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:15.811237 14199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:15.812019 14193 server_base.cc:1034] running on GCE node
I20260923 20:47:15.812865 14193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:15.814316 14193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:15.815510 14193 hybrid_clock.cc:648] HybridClock initialized: now 1790196435815465 us; error 36 us; skew 500 ppm
I20260923 20:47:15.815917 14193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:15 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196435, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:15.818964 14193 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:47:15.820603 14193 webserver.cc:460] Webserver started at http://127.5.71.190:35301/ using document root <none> and password file <none>
I20260923 20:47:15.821267 14193 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:15.821386 14193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:15.826975 14193 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 20:47:15.829934 14208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:15.831364 14193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:47:15.831552 14193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "bd91bb8101a14936927f21b59ede20c2"
format_stamp: "Formatted at 2026-09-23 20:47:12 on dist-test-slave-dn29"
server_key: "fc3af0d7fa03d1ade8b02278b61d527a"
server_key_iv: "95fe7bc5ec70e4e491c59654e6720b8b"
server_key_version: "encryptionkey@0"
I20260923 20:47:15.832582 14193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:15.853348 14193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:15.857613 14193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:15.857923 14193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:15.878108 14193 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45569
I20260923 20:47:15.878137 14259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45569 every 8 connection(s)
I20260923 20:47:15.879673 14193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:47:15.886976 14260 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:47:15.887878 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14193
I20260923 20:47:15.888733 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:40927
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:45569
--webserver_port=40391
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:47:15.890707 14260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Bootstrap starting.
I20260923 20:47:15.917516 14260 log.cc:824] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:15.929986 14260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:47:15.930732 14260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Bootstrap complete.
I20260923 20:47:15.945247 14260 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:15.946697 14260 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd91bb8101a14936927f21b59ede20c2, State: Initialized, Role: FOLLOWER
I20260923 20:47:15.947499 14260 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:15.947866 14260 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:47:15.948009 14260 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:47:15.948213 14260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 20:47:15.954530 14260 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:15.955248 14260 leader_election.cc:302] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd91bb8101a14936927f21b59ede20c2; no voters:
I20260923 20:47:15.956162 14260 leader_election.cc:288] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 20:47:15.956367 14264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 20:47:15.958256 14264 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [term 2 LEADER]: Becoming Leader. State: Replica: bd91bb8101a14936927f21b59ede20c2, State: Running, Role: LEADER
I20260923 20:47:15.959014 14264 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } }
I20260923 20:47:15.959281 14260 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:47:15.961768 14266 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd91bb8101a14936927f21b59ede20c2. Latest consensus state: current_term: 2 leader_uuid: "bd91bb8101a14936927f21b59ede20c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } } }
I20260923 20:47:15.962189 14266 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:15.962704 14265 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bd91bb8101a14936927f21b59ede20c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd91bb8101a14936927f21b59ede20c2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45569 } } }
I20260923 20:47:15.963044 14265 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:15.973485 14270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:15.981230 14270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:15.982779 14270 catalog_manager.cc:1259] Loaded cluster ID: cc7de454cf954f708b72042a03a3f0a2
I20260923 20:47:15.982954 14270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:15.987094 14270 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:15.988756 14270 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd91bb8101a14936927f21b59ede20c2: Loaded TSK: 0
I20260923 20:47:15.990837 14270 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 20:47:16.273262 14262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:16.273671 14262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:16.273844 14262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:16.282274 14262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:16.282477 14262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:16.282706 14262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:47:16.293743 14262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.129.keytab
--principal=oryx/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:40927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=40391
--tserver_master_addrs=127.5.71.190:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:16.295239 14262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:16.296756 14262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:16.313262 14287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:16.313374 14289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:16.313943 14286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:16.314375 14262 server_base.cc:1034] running on GCE node
I20260923 20:47:16.315021 14262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:16.316130 14262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:16.317394 14262 hybrid_clock.cc:648] HybridClock initialized: now 1790196436317323 us; error 58 us; skew 500 ppm
I20260923 20:47:16.317811 14262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:16.321651 14262 init.cc:377] Logged in from keytab as oryx/127.5.71.129@KRBTEST.COM (short username oryx)
I20260923 20:47:16.323410 14262 webserver.cc:460] Webserver started at http://127.5.71.129:40391/ using document root <none> and password file <none>
I20260923 20:47:16.324265 14262 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:16.324426 14262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:16.330489 14262 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 20:47:16.334043 14296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:16.335595 14262 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260923 20:47:16.335780 14262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "c1c3a739a8d148d3a493a72716ff3580"
format_stamp: "Formatted at 2026-09-23 20:47:13 on dist-test-slave-dn29"
server_key: "454f22053d3ccd9b6cc834626113ea99"
server_key_iv: "b7f033493460edc60bd07ba7019cabe5"
server_key_version: "encryptionkey@0"
I20260923 20:47:16.336890 14262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:16.350620 14262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:16.355125 14262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:16.355432 14262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:16.356555 14262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:16.358006 14262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:16.358094 14262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:16.358218 14262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:16.358309 14262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:16.393317 14262 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:40927
I20260923 20:47:16.393412 14408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:40927 every 8 connection(s)
I20260923 20:47:16.394968 14262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:47:16.397507 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14262
I20260923 20:47:16.398206 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:46221
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:45569
--webserver_port=43177
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:16.416716 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:16.398561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:46003)
0923 20:47:16.399724 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:16.399773 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:16.401570 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:16.403400 (+ 1830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:16.403409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:16.404119 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:16.404705 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.404724 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.406510 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.406519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.408831 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.408852 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.410111 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.410121 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.410441 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.411263 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:16.411299 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:16.415631 (+ 4332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":869,"thread_start_us":676,"threads_started":1}
W20260923 20:47:16.417469 14409 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45569 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.71.190:45569: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
W20260923 20:47:16.417539 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:16.398436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:46003 (local address 127.5.71.190:45569)
0923 20:47:16.399296 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:16.399326 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:16.399961 (+ 635us) server_negotiation.cc:407] Connection header received
0923 20:47:16.402222 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:16.402247 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:16.402776 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:16.403224 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:16.404961 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.406302 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.409028 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.409843 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.410002 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.416212 (+ 6210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:46003: BlockingRecv error: recv got EOF from 127.5.71.129:46003 (error 108)
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":308,"threads_started":1}
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:16.426203 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:16.418075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:40339)
0923 20:47:16.418347 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:16.418369 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:16.418479 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:16.418972 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:16.418976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:16.419285 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:16.419426 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.419434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.420545 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.420551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.421496 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.421507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.422222 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.422227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.422416 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.423021 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:16.423038 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:16.425907 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143}
W20260923 20:47:16.426354 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:16.418382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40339 (local address 127.5.71.190:45569)
0923 20:47:16.418591 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:16.418599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:16.418623 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:47:16.418693 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:16.418700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:16.418788 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:16.418923 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:16.419584 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.420404 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.421647 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.422058 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.422213 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.426118 (+ 3905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:40339: BlockingRecv error: recv got EOF from 127.5.71.129:40339 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:16.436058 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:16.426996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:34329)
0923 20:47:16.427221 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:16.427241 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:16.427339 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:16.427895 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:16.427899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:16.428185 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:16.428331 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.428338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.429853 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.429857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.430939 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.430951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.431921 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.431926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.432050 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.432757 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:16.432776 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:16.435639 (+ 2863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126}
W20260923 20:47:16.436395 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:16.427046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:34329 (local address 127.5.71.190:45569)
0923 20:47:16.427204 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:16.427209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:16.427333 (+ 124us) server_negotiation.cc:407] Connection header received
0923 20:47:16.427462 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:16.427467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:16.427543 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:16.427645 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:16.428468 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.429688 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.431085 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.431715 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.431873 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.435975 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:34329: BlockingRecv error: recv got EOF from 127.5.71.129:34329 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260923 20:47:16.436452 14409 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 20:47:16.747737 14413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:16.748091 14413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:16.748219 14413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:16.756384 14413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:16.756507 14413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:16.756623 14413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:47:16.767062 14413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.130.keytab
--principal=oryx/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:46221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=43177
--tserver_master_addrs=127.5.71.190:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:16.768587 14413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:16.769927 14413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:16.785817 14418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:16.785771 14421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:16.785735 14419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:16.786612 14413 server_base.cc:1034] running on GCE node
I20260923 20:47:16.787415 14413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:16.788587 14413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:16.789885 14413 hybrid_clock.cc:648] HybridClock initialized: now 1790196436789812 us; error 71 us; skew 500 ppm
I20260923 20:47:16.790270 14413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196436, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:16.794193 14413 init.cc:377] Logged in from keytab as oryx/127.5.71.130@KRBTEST.COM (short username oryx)
I20260923 20:47:16.796200 14413 webserver.cc:460] Webserver started at http://127.5.71.130:43177/ using document root <none> and password file <none>
I20260923 20:47:16.796952 14413 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:16.797070 14413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:16.802605 14413 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260923 20:47:16.805572 14428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:16.807449 14413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:47:16.807622 14413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "c4de0dbde72b40fd8190c6425e4235e7"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1b34d625852b0727fcb4dc33dabfcfe1"
server_key_iv: "1f699fa757566a8acb864abf0f8c2f7f"
server_key_version: "encryptionkey@0"
I20260923 20:47:16.808687 14413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:16.822189 14413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:16.826876 14413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:16.827184 14413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:16.828439 14413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:16.829802 14413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:16.829890 14413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:16.830054 14413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:16.830138 14413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:16.866457 14413 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:46221
I20260923 20:47:16.866503 14540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:46221 every 8 connection(s)
I20260923 20:47:16.868005 14413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:47:16.874583 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14413
I20260923 20:47:16.875325 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:43687
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:45569
--webserver_port=33209
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:34719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 20:47:16.886794 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:16.870652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:40111 (local address 127.5.71.190:45569)
0923 20:47:16.870877 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:16.870885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:16.872327 (+ 1442us) server_negotiation.cc:407] Connection header received
0923 20:47:16.874183 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:16.874189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:16.874319 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:16.874481 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:16.876577 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.877738 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.880191 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.881029 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.881176 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.886560 (+ 5384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:40111: BlockingRecv error: recv got EOF from 127.5.71.130:40111 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260923 20:47:16.887593 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:16.871217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:40111)
0923 20:47:16.872144 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:16.872188 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:16.873861 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:16.874889 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:16.874898 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:16.875638 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:16.876352 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.876371 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.877965 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.877971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.880010 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.880030 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.881287 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.881295 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.881580 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.882424 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:16.882453 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:16.886378 (+ 3925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":327,"threads_started":1}
W20260923 20:47:16.888329 14541 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45569 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.71.190:45569: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:16.896664 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:16.888940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:33647)
0923 20:47:16.889189 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:16.889208 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:16.889310 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:16.889645 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:16.889649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:16.889976 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:16.890115 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.890122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.891110 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.891115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.892089 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.892099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.892799 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.892804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.892973 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.893557 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:16.893573 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:16.896373 (+ 2800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133}
W20260923 20:47:16.896840 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:16.889084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33647 (local address 127.5.71.190:45569)
0923 20:47:16.889249 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:16.889255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:16.889272 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:47:16.889459 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:16.889463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:16.889520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:16.889622 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:16.890255 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.890975 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.892235 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.892642 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.892788 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.896602 (+ 3814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:33647: BlockingRecv error: recv got EOF from 127.5.71.130:33647 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:16 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:16.906044 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:16.897467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:57563)
0923 20:47:16.897632 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:16.897650 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:16.897736 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:16.898104 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:16.898107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:16.898409 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:16.898547 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.898555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.900139 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.900143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.901034 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:16.901044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.902078 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.902082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:16.902198 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.902882 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:16.902899 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:16.905827 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260923 20:47:16.906210 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:16.897563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:57563 (local address 127.5.71.190:45569)
0923 20:47:16.897773 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:16.897782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:16.897804 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:47:16.897865 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:16.897873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:16.897960 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:16.898075 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:16.898709 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.899997 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.901174 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:16.901931 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:16.902080 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:16.906000 (+ 3920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:57563: BlockingRecv error: recv got EOF from 127.5.71.130:57563 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260923 20:47:16.906493 14541 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 20:47:17.215432 14544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:17.215826 14544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:17.216009 14544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:17.223399 14544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:17.223564 14544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:17.223762 14544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:17.233711 14544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/krb5kdc/oryx_127.5.71.131.keytab
--principal=oryx/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:43687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=33209
--tserver_master_addrs=127.5.71.190:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:17.235131 14544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:17.236662 14544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:17.250838 14552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:17.250857 14549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:17.250838 14550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:17.251530 14544 server_base.cc:1034] running on GCE node
I20260923 20:47:17.252275 14544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:17.253269 14544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:17.254519 14544 hybrid_clock.cc:648] HybridClock initialized: now 1790196437254478 us; error 40 us; skew 500 ppm
I20260923 20:47:17.254873 14544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196437, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:17.258248 14544 init.cc:377] Logged in from keytab as oryx/127.5.71.131@KRBTEST.COM (short username oryx)
I20260923 20:47:17.259905 14544 webserver.cc:460] Webserver started at http://127.5.71.131:33209/ using document root <none> and password file <none>
I20260923 20:47:17.260703 14544 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:17.260866 14544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:17.266601 14544 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260923 20:47:17.269836 14559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:17.271454 14544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:47:17.271656 14544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "e69d59e384d34bc4a8b60e79db50678f"
format_stamp: "Formatted at 2026-09-23 20:47:14 on dist-test-slave-dn29"
server_key: "1e289d4f06b0026c785e1b0bcf7bb6e7"
server_key_iv: "cecc888a583099a4fecf370c629fc341"
server_key_version: "encryptionkey@0"
I20260923 20:47:17.272851 14544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:17.300307 14544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:17.305469 14544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:17.305833 14544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:17.307168 14544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:17.308804 14544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:17.308902 14544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:17.308974 14544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:17.309029 14544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:17.346077 14544 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:43687
I20260923 20:47:17.346194 14671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:43687 every 8 connection(s)
I20260923 20:47:17.347587 14544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:17.355365 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14544
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 20:47:17.367395 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:17.350490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:60757 (local address 127.5.71.190:45569)
0923 20:47:17.350727 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:17.350735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:17.351992 (+ 1257us) server_negotiation.cc:407] Connection header received
0923 20:47:17.354080 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:17.354085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:17.354180 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:17.354276 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:17.356313 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.357447 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.359429 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.360134 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.360274 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.367084 (+ 6810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:60757: BlockingRecv error: recv got EOF from 127.5.71.131:60757 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:47:17.368552 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:17.350987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:60757)
0923 20:47:17.351826 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:17.351869 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:17.353769 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:17.354570 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:17.354580 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:17.355215 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:17.356021 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.356051 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.357603 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.357612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.359136 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.359149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.360387 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.360395 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.360761 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.361561 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:17.361597 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:17.366883 (+ 5286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":322,"threads_started":1}
W20260923 20:47:17.369455 14672 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45569 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.71.190:45569: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
I20260923 20:47:17.374703 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:17.356599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.5.71.190:45569)
0923 20:47:17.357406 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:17.357411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:17.357430 (+ 19us) server_negotiation.cc:407] Connection header received
0923 20:47:17.357475 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:17.357478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:17.357579 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:17.357714 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:17.359334 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.360454 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.361939 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.362717 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.362884 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.366038 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:17.366055 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:17.366083 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:17.366153 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:17.370671 (+ 4518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:17.371532 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:17.371546 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:17.371565 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:17.371784 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:17.372346 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:17.372354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:17.372359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:17.372888 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:17.373294 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:17.373858 (+ 564us) server_negotiation.cc:299] Negotiation successful
0923 20:47:17.374277 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":579,"threads_started":1}
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 20:47:17.379727 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:17.370276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:55067 (local address 127.5.71.190:45569)
0923 20:47:17.370437 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:17.370443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:17.370461 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:47:17.370558 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:17.370562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:17.370645 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:17.370761 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:17.371418 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.372265 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.373454 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.374057 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.374203 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.379513 (+ 5310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:55067: BlockingRecv error: recv got EOF from 127.5.71.131:55067 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 20:47:17.379930 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:17.370153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:55067)
0923 20:47:17.370324 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:17.370344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:17.370450 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:17.370781 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:17.370785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:17.371115 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:17.371255 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.371263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.372416 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.372421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.373311 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.373322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.374391 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.374395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.374546 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.375284 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:17.375306 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:17.379211 (+ 3905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:17.388634 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:17.380536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:59239)
0923 20:47:17.380722 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:17.380751 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:17.380878 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:17.381195 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:17.381200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:17.381515 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:17.381658 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.381667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.382695 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.382702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.383629 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.383640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.384376 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.384382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.384560 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.385161 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:17.385179 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:17.388370 (+ 3191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260923 20:47:17.389048 14672 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 20:47:17.389073 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:17.380694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:59239 (local address 127.5.71.190:45569)
0923 20:47:17.380939 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:17.380945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:17.380963 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:47:17.381016 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:17.381019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:17.381071 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:17.381172 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:17.381801 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.382560 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.383775 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.384226 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.384367 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.388627 (+ 4260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:59239: BlockingRecv error: recv got EOF from 127.5.71.131:59239 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:17.446770 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:17.437227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:51925)
0923 20:47:17.437824 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:17.437841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:17.438010 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:17.438400 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:17.438404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:17.438908 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:17.439102 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.439122 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.440251 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.440257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.441336 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.441348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.442278 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.442286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.442454 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.443121 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:17.443139 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:17.446417 (+ 3278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":325,"threads_started":1}
W20260923 20:47:17.446892 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:17.437289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:51925 (local address 127.5.71.190:45569)
0923 20:47:17.437583 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:17.437593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:17.437941 (+ 348us) server_negotiation.cc:407] Connection header received
0923 20:47:17.438158 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:17.438165 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:17.438255 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:17.438368 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:17.439272 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.440100 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.441479 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.442073 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.442215 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.446658 (+ 4443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:51925: BlockingRecv error: recv got EOF from 127.5.71.129:51925 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:17 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:17.917743 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:17.907250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:38287)
0923 20:47:17.907773 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:17.907808 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:17.907931 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:17.908271 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:17.908274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:17.908831 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:17.909031 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.909040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.910124 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.910128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.911073 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:17.911084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.911965 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.911972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:17.912252 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.913090 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:17.913113 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:17.917380 (+ 4267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":242,"threads_started":1}
W20260923 20:47:17.917865 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:17.907330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:38287 (local address 127.5.71.190:45569)
0923 20:47:17.907566 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:17.907573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:17.907921 (+ 348us) server_negotiation.cc:407] Connection header received
0923 20:47:17.908033 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:17.908037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:17.908122 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:17.908216 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:17.909199 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.909983 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.911218 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:17.911682 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:17.911778 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:17.917607 (+ 5829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:38287: BlockingRecv error: recv got EOF from 127.5.71.130:38287 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 23 20:47:18 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:18 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:18.404999 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:18.389638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:50425)
0923 20:47:18.390177 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:18.390194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:18.390316 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:18.390676 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:18.390680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:18.391207 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:18.391424 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:18.391435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.392636 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.392640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:18.393782 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:18.393797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.394828 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.394834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:18.394962 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:18.401668 (+ 6706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:18.401692 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:18.404755 (+ 3063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":283,"threads_started":1}
W20260923 20:47:18.405362 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:18.389696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:50425 (local address 127.5.71.190:45569)
0923 20:47:18.389866 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:18.389870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:18.390268 (+ 398us) server_negotiation.cc:407] Connection header received
0923 20:47:18.390418 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:18.390422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:18.390484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:18.390567 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:18.391596 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.392472 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.393954 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.394587 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.394717 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:18.405029 (+ 10312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:50425: BlockingRecv error: recv got EOF from 127.5.71.131:50425 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 23 20:47:18 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:18 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:18.457458 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:18.447981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:46565)
0923 20:47:18.448546 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:18.448563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:18.448705 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:18.449086 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:18.449089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:18.449624 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:18.449803 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:18.449813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.451036 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.451041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:18.452048 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:18.452061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.452793 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.452800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:18.452965 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:18.453830 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:18.453848 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:18.457095 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":348,"threads_started":1}
W20260923 20:47:18.457604 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:18.448190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:46565 (local address 127.5.71.190:45569)
0923 20:47:18.448457 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:18.448464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:18.448658 (+ 194us) server_negotiation.cc:407] Connection header received
0923 20:47:18.448839 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:18.448844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:18.448934 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:18.449060 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:18.449958 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.450896 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.452188 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.452593 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.452759 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:18.457340 (+ 4581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:46565: BlockingRecv error: recv got EOF from 127.5.71.129:46565 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Sep 23 20:47:18 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:18 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:18.928417 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:18.918682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:51831)
0923 20:47:18.919179 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:18.919197 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:18.919324 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:18.919709 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:18.919713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:18.920237 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:18.920391 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:18.920400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.921666 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.921671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:18.922728 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:18.922740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.923494 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.923499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:18.923663 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:18.924416 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:18.924435 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:18.928143 (+ 3708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":270,"threads_started":1}
W20260923 20:47:18.928515 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:18.918887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:51831 (local address 127.5.71.190:45569)
0923 20:47:18.919124 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:18.919137 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:18.919280 (+ 143us) server_negotiation.cc:407] Connection header received
0923 20:47:18.919438 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:18.919442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:18.919502 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:18.919613 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:18.920606 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.921467 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.922881 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:18.923338 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:18.923443 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:18.928339 (+ 4896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:51831: BlockingRecv error: recv got EOF from 127.5.71.130:51831 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 23 20:47:19 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:19 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:19.415632 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:19.406368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:46553)
0923 20:47:19.407059 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:19.407080 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:19.407209 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:19.407605 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:19.407611 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:19.408134 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:19.408292 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:19.408302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.409422 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.409429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:19.410437 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:19.410449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.411263 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.411269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:19.411473 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:19.412154 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:19.412173 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:19.415251 (+ 3078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":442,"threads_started":1}
W20260923 20:47:19.415768 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:19.406636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:46553 (local address 127.5.71.190:45569)
0923 20:47:19.406926 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:19.406933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:19.407207 (+ 274us) server_negotiation.cc:407] Connection header received
0923 20:47:19.407363 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:19.407367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:19.407433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:19.407555 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:19.408453 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.409270 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.410600 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.411067 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.411248 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:19.415483 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:46553: BlockingRecv error: recv got EOF from 127.5.71.131:46553 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Sep 23 20:47:19 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:19 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:19.467726 14683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:19.458605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:59625)
0923 20:47:19.459103 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:19.459147 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:19.459274 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:19.459769 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:19.459779 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:19.460273 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:19.460443 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:19.460452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.461782 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.461787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:19.462882 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:19.462895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.463614 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.463620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:19.463909 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:19.464586 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:19.464605 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:19.467410 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":275,"threads_started":1}
W20260923 20:47:19.468068 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:19.458717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:59625 (local address 127.5.71.190:45569)
0923 20:47:19.458941 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:19.458948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:19.459215 (+ 267us) server_negotiation.cc:407] Connection header received
0923 20:47:19.459392 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:19.459399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:19.459493 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:19.459608 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:19.460601 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.461555 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.463046 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.463437 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.463531 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:19.467689 (+ 4158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:59625: BlockingRecv error: recv got EOF from 127.5.71.129:59625 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 23 20:47:19 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:19 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:19.939060 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:19.929498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:46045)
0923 20:47:19.930078 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:19.930096 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:19.930276 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:19.930651 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:19.930656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:19.931071 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:19.931263 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:19.931273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.932536 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.932542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:19.933548 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:19.933560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.934335 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.934342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:19.934541 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:19.935191 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:19.935211 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:19.938544 (+ 3333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":365,"threads_started":1}
W20260923 20:47:19.939131 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:19.929549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:46045 (local address 127.5.71.190:45569)
0923 20:47:19.929768 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:19.929776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:19.930229 (+ 453us) server_negotiation.cc:407] Connection header received
0923 20:47:19.930431 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:19.930441 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:19.930507 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:19.930618 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:19.931454 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.932376 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.933701 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:19.934159 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:19.934319 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:19.938857 (+ 4538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:46045: BlockingRecv error: recv got EOF from 127.5.71.130:46045 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 20:47:20 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:20 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:20.426692 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:20.416855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:43867)
0923 20:47:20.417484 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:20.417502 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:20.417624 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:20.417973 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:20.417978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:20.418375 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:20.418551 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:20.418560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.419594 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.419598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:20.420818 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:20.420832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.421524 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.421529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:20.421663 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:20.422421 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:20.422440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:20.426353 (+ 3913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":389,"threads_started":1}
W20260923 20:47:20.426931 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:20.417038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:43867 (local address 127.5.71.190:45569)
0923 20:47:20.417273 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:20.417278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:20.417590 (+ 312us) server_negotiation.cc:407] Connection header received
0923 20:47:20.417737 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:20.417743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:20.417827 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:20.417916 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:20.418698 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.419457 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.420961 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.421369 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.421465 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:20.426619 (+ 5154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:43867: BlockingRecv error: recv got EOF from 127.5.71.131:43867 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 23 20:47:20 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:20 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:20.479055 14686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:20.469173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:56543)
0923 20:47:20.469720 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:20.469740 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:20.469874 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:20.470245 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:20.470249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:20.470760 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:20.470950 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:20.470962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.472184 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.472189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:20.473260 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:20.473273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.474305 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.474313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:20.474560 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:20.475437 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:20.475456 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:20.478795 (+ 3339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":286,"threads_started":1}
W20260923 20:47:20.479449 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:20.469286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:56543 (local address 127.5.71.190:45569)
0923 20:47:20.469583 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:20.469589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:20.469852 (+ 263us) server_negotiation.cc:407] Connection header received
0923 20:47:20.469967 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:20.469971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:20.470033 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:20.470116 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:20.471170 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.472034 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.473433 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.474056 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.474200 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:20.479004 (+ 4804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:56543: BlockingRecv error: recv got EOF from 127.5.71.129:56543 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Sep 23 20:47:20 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:20 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:20.949885 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:20.940300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:36751)
0923 20:47:20.940827 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:20.940844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:20.940964 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:20.941273 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:20.941276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:20.941752 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:20.941902 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:20.941911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.942984 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.942988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:20.944214 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:20.944227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.945278 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.945284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:20.945509 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:20.946217 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:20.946239 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:20.949509 (+ 3270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":276,"threads_started":1}
W20260923 20:47:20.950088 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:20.940427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:36751 (local address 127.5.71.190:45569)
0923 20:47:20.940675 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:20.940682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:20.940930 (+ 248us) server_negotiation.cc:407] Connection header received
0923 20:47:20.941067 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:20.941071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:20.941134 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:20.941218 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:20.942071 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.942832 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.944387 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:20.945089 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:20.945269 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:20.949829 (+ 4560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:36751: BlockingRecv error: recv got EOF from 127.5.71.130:36751 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 23 20:47:21 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:21 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:21.436978 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:21.427691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:58265)
0923 20:47:21.428319 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:21.428344 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:21.428477 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:21.428832 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:21.428836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:21.429240 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:21.429395 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:21.429404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.430546 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.430553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:21.431679 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:21.431694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.432547 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.432551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:21.432698 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:21.433399 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:21.433416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:21.436698 (+ 3282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":283,"threads_started":1}
W20260923 20:47:21.437238 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:21.427865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:58265 (local address 127.5.71.190:45569)
0923 20:47:21.428098 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:21.428104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:21.428419 (+ 315us) server_negotiation.cc:407] Connection header received
0923 20:47:21.428592 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:21.428595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:21.428664 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:21.428741 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:21.429582 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.430373 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.431905 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.432391 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.432499 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:21.436939 (+ 4440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:58265: BlockingRecv error: recv got EOF from 127.5.71.131:58265 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 23 20:47:21 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:21 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:21.489355 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:21.480059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:55409)
0923 20:47:21.480584 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:21.480606 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:21.480754 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:21.481177 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:21.481181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:21.481604 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:21.481800 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:21.481811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.482919 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.482924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:21.484012 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:21.484027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.484903 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.484908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:21.485069 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:21.485804 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:21.485823 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:21.489033 (+ 3210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":256,"threads_started":1}
W20260923 20:47:21.489451 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:21.480269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:55409 (local address 127.5.71.190:45569)
0923 20:47:21.480531 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:21.480536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:21.480726 (+ 190us) server_negotiation.cc:407] Connection header received
0923 20:47:21.480917 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:21.480921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:21.480985 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:21.481074 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:21.481963 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.482763 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.484199 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.484733 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.484844 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:21.489274 (+ 4430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:55409: BlockingRecv error: recv got EOF from 127.5.71.129:55409 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 23 20:47:21 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:21 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:21.960922 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:21.951072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:33527)
0923 20:47:21.951731 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:21.951749 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:21.951977 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:21.952341 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:21.952344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:21.952832 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:21.953010 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:21.953020 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.954056 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.954060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:21.955176 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:21.955194 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.956181 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.956188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:21.956352 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:21.957000 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:21.957028 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:21.960642 (+ 3614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":324,"threads_started":1}
W20260923 20:47:21.961220 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:21.951362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33527 (local address 127.5.71.190:45569)
0923 20:47:21.951637 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:21.951644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:21.951908 (+ 264us) server_negotiation.cc:407] Connection header received
0923 20:47:21.952129 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:21.952133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:21.952198 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:21.952278 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:21.953163 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.953919 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.955384 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:21.955975 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:21.956133 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:21.960958 (+ 4825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:33527: BlockingRecv error: recv got EOF from 127.5.71.130:33527 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 23 20:47:22 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:22 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:22.447520 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:22.438199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:52705)
0923 20:47:22.438758 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:22.438775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:22.438889 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:22.439249 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:22.439254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:22.439643 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:22.439949 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:22.439963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.441427 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.441431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:22.442598 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:22.442612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.443330 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.443336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:22.443502 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:22.444196 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:22.444217 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:22.447166 (+ 2949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":273,"threads_started":1}
W20260923 20:47:22.447646 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:22.438280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:52705 (local address 127.5.71.190:45569)
0923 20:47:22.438496 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:22.438501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:22.438902 (+ 401us) server_negotiation.cc:407] Connection header received
0923 20:47:22.439005 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:22.439009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:22.439072 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:22.439165 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:22.440128 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.441222 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.442743 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.443179 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.443324 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:22.447397 (+ 4073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:52705: BlockingRecv error: recv got EOF from 127.5.71.131:52705 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 23 20:47:22 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:22 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:22.500803 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:22.490708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:43411)
0923 20:47:22.491439 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:22.491460 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:22.491668 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:22.492237 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:22.492246 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:22.492792 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:22.493052 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:22.493066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.494531 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.494537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:22.495621 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:22.495636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.496568 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.496573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:22.496790 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:22.497546 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:22.497564 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:22.500510 (+ 2946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":463,"threads_started":1}
W20260923 20:47:22.501180 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:22.491070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:43411 (local address 127.5.71.190:45569)
0923 20:47:22.491386 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:22.491395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:22.491583 (+ 188us) server_negotiation.cc:407] Connection header received
0923 20:47:22.491977 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:22.491981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:22.492049 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:22.492155 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:22.493276 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.494337 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.495841 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.496379 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.496503 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:22.500738 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:43411: BlockingRecv error: recv got EOF from 127.5.71.129:43411 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 23 20:47:22 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:22 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:22.972105 14693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:22.962240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:44521)
0923 20:47:22.962903 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:22.962921 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:22.963047 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:22.963425 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:22.963429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:22.963997 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:22.964171 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:22.964180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.965662 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.965668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:22.966843 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:22.966856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.967842 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.967851 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:22.967986 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:22.968723 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:22.968744 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:22.971878 (+ 3134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":350,"threads_started":1}
W20260923 20:47:22.972453 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:22.962347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:44521 (local address 127.5.71.190:45569)
0923 20:47:22.962561 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:22.962566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:22.963023 (+ 457us) server_negotiation.cc:407] Connection header received
0923 20:47:22.963165 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:22.963169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:22.963237 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:22.963325 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:22.964343 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.965431 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.966995 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:22.967555 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:22.967762 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:22.972180 (+ 4418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:44521: BlockingRecv error: recv got EOF from 127.5.71.130:44521 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 23 20:47:23 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:23 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:23.466423 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:23.448900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:39095)
0923 20:47:23.449606 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:23.449625 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:23.449762 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:23.450110 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:23.450114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:23.450547 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:23.450784 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:23.450799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.452333 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.452462 (+ 129us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:23.453747 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:23.453761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.454626 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.454631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:23.454776 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:23.462822 (+ 8046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:23.462846 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:23.466196 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":436,"threads_started":1}
W20260923 20:47:23.466775 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:23.448957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:39095 (local address 127.5.71.190:45569)
0923 20:47:23.449177 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:23.449182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:23.449761 (+ 579us) server_negotiation.cc:407] Connection header received
0923 20:47:23.449873 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:23.449877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:23.449943 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:23.450038 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:23.451003 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.452161 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.453923 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.454441 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.454561 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:23.466447 (+ 11886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:39095: BlockingRecv error: recv got EOF from 127.5.71.131:39095 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 23 20:47:23 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:23 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:23.512235 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:23.502152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:35749)
0923 20:47:23.502919 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:23.502943 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:23.503145 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:23.503558 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:23.503563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:23.504240 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:23.504451 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:23.504462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.505581 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.505589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:23.506635 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:23.506648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.507443 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.507457 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:23.507646 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:23.508391 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:23.508410 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:23.511949 (+ 3539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":306,"threads_started":1}
W20260923 20:47:23.512420 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:23.502352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:35749 (local address 127.5.71.190:45569)
0923 20:47:23.502586 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:23.502595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:23.503022 (+ 427us) server_negotiation.cc:407] Connection header received
0923 20:47:23.503297 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:23.503305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:23.503395 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:23.503519 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:23.504623 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.505420 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.506809 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.507269 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.507423 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:23.512170 (+ 4747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:35749: BlockingRecv error: recv got EOF from 127.5.71.129:35749 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 23 20:47:23 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:23 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:23.982801 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:23.973196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:46071)
0923 20:47:23.974047 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:23.974065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:23.974265 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:23.974763 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:23.974769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:23.975260 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:23.975430 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:23.975440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.976661 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.976667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:23.977713 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:23.977724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.978513 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.978519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:23.978716 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:23.979307 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:23.979327 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:23.982504 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":478,"threads_started":1}
W20260923 20:47:23.982991 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:23.973364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:46071 (local address 127.5.71.190:45569)
0923 20:47:23.973599 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:23.973603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:23.974218 (+ 615us) server_negotiation.cc:407] Connection header received
0923 20:47:23.974486 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:23.974491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:23.974567 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:23.974733 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:23.975590 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.976501 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.977872 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:23.978329 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:23.978501 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:23.982734 (+ 4233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:46071: BlockingRecv error: recv got EOF from 127.5.71.130:46071 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 23 20:47:24 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:24 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:24.476653 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:24.467436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:41189)
0923 20:47:24.468144 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:24.468161 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:24.468279 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:24.468598 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:24.468601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:24.469049 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:24.469271 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:24.469280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.470325 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.470329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:24.471244 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:24.471255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.472216 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.472221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:24.472379 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:24.473039 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:24.473056 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:24.476368 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":408,"threads_started":1}
W20260923 20:47:24.476959 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:24.467492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:41189 (local address 127.5.71.190:45569)
0923 20:47:24.467689 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:24.467694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:24.468255 (+ 561us) server_negotiation.cc:407] Connection header received
0923 20:47:24.468383 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:24.468387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:24.468450 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:24.468534 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:24.469406 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.470178 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.471410 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.472045 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.472162 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:24.476685 (+ 4523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:41189: BlockingRecv error: recv got EOF from 127.5.71.131:41189 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 23 20:47:24 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:24 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:24.524998 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:24.513302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:35899)
0923 20:47:24.513980 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:24.514007 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:24.514163 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:24.514691 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:24.514696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:24.515396 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:24.515640 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:24.515653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.517470 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.517475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:24.519093 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:24.519112 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.520424 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.520431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:24.520644 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:24.521365 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:24.521390 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:24.524747 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":429,"threads_started":1}
W20260923 20:47:24.525734 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:24.513708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:35899 (local address 127.5.71.190:45569)
0923 20:47:24.514050 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:24.514059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:24.514128 (+ 69us) server_negotiation.cc:407] Connection header received
0923 20:47:24.514335 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:24.514344 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:24.514494 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:24.514635 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:24.515892 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.517264 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.519277 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.520157 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.520323 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:24.525019 (+ 4696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:35899: BlockingRecv error: recv got EOF from 127.5.71.129:35899 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Sep 23 20:47:24 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:24 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:24.994019 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:24.983993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:44445)
0923 20:47:24.984618 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:24.984645 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:24.984780 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:24.985175 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:24.985179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:24.985602 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:24.985764 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:24.985772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.987023 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.987028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:24.988129 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:24.988145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.989178 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.989193 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:24.989372 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:24.990041 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:24.990071 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:24.993667 (+ 3596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":347,"threads_started":1}
W20260923 20:47:24.994195 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:24.984106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:44445 (local address 127.5.71.190:45569)
0923 20:47:24.984411 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:24.984416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:24.984739 (+ 323us) server_negotiation.cc:407] Connection header received
0923 20:47:24.984893 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:24.984897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:24.984969 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:24.985070 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:24.985965 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.986841 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.988363 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:24.988959 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:24.989122 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:24.993957 (+ 4835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:44445: BlockingRecv error: recv got EOF from 127.5.71.130:44445 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
Sep 23 20:47:25 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:25 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:25.488147 14700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:25.477652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:50265)
0923 20:47:25.478351 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:25.478367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:25.478491 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:25.478888 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:25.478893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:25.479375 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:25.479534 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:25.479543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.481364 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.481376 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:25.482466 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:25.482480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.483435 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.483441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:25.483581 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:25.484332 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:25.484352 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:25.487777 (+ 3425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":427,"threads_started":1}
W20260923 20:47:25.488418 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:25.477955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:50265 (local address 127.5.71.190:45569)
0923 20:47:25.478201 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:25.478207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:25.478471 (+ 264us) server_negotiation.cc:407] Connection header received
0923 20:47:25.478601 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:25.478605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:25.478677 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:25.478790 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:25.479725 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.481086 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.482639 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.483196 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.483322 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:25.488139 (+ 4817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:50265: BlockingRecv error: recv got EOF from 127.5.71.131:50265 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 20:47:25 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:25 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:25.535620 14701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:25.526228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:37659)
0923 20:47:25.526919 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:25.526934 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:25.527050 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:25.527465 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:25.527469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:25.528043 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:25.528212 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:25.528222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.529339 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.529343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:25.530317 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:25.530329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.531261 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.531267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:25.531460 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:25.532210 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:25.532234 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:25.535372 (+ 3138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":385,"threads_started":1}
W20260923 20:47:25.535887 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:25.526348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:37659 (local address 127.5.71.190:45569)
0923 20:47:25.526531 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:25.526536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:25.527034 (+ 498us) server_negotiation.cc:407] Connection header received
0923 20:47:25.527161 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:25.527166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:25.527256 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:25.527354 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:25.528354 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.529199 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.530510 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.531075 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.531213 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:25.535590 (+ 4377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:37659: BlockingRecv error: recv got EOF from 127.5.71.129:37659 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 23 20:47:25 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:25 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:26.004715 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:25.994946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:33853)
0923 20:47:25.995548 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:25.995564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:25.995678 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:25.996085 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:25.996089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:25.996538 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:25.996823 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:25.996836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.998000 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.998004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:25.998984 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:25.998995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.999891 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.999898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:26.000066 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:26.000684 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:26.000705 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:26.004363 (+ 3658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":361,"threads_started":1}
W20260923 20:47:26.004904 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:25.995034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33853 (local address 127.5.71.190:45569)
0923 20:47:25.995276 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:25.995280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:25.995665 (+ 385us) server_negotiation.cc:407] Connection header received
0923 20:47:25.995828 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:25.995833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:25.995898 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:25.995987 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:25.997005 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.997846 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.999174 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:25.999622 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:25.999743 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:26.004632 (+ 4889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:33853: BlockingRecv error: recv got EOF from 127.5.71.130:33853 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 23 20:47:26 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:26 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:26.498387 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:26.489318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:36977)
0923 20:47:26.490110 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:26.490126 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:26.490309 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:26.490673 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:26.490682 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:26.491087 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:26.491251 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:26.491260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.492379 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.492385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:26.493344 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:26.493356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.494115 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.494121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:26.494311 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:26.494996 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:26.495015 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:26.498036 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":548,"threads_started":1}
W20260923 20:47:26.498544 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:26.489391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:36977 (local address 127.5.71.190:45569)
0923 20:47:26.489647 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:26.489667 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:26.490230 (+ 563us) server_negotiation.cc:407] Connection header received
0923 20:47:26.490439 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:26.490445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:26.490524 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:26.490626 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:26.491413 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.492252 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.493496 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.493950 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.494097 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:26.498262 (+ 4165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:36977: BlockingRecv error: recv got EOF from 127.5.71.131:36977 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 23 20:47:26 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:26 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:26.545892 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:26.536767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:40771)
0923 20:47:26.537401 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:26.537417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:26.537583 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:26.537937 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:26.537941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:26.538344 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:26.538497 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:26.538506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.539605 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.539611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:26.540724 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:26.540737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.541486 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.541492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:26.541678 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:26.542278 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:26.542298 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:26.545445 (+ 3147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":410,"threads_started":1}
W20260923 20:47:26.545923 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:26.536752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40771 (local address 127.5.71.190:45569)
0923 20:47:26.537016 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:26.537021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:26.537507 (+ 486us) server_negotiation.cc:407] Connection header received
0923 20:47:26.537714 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:26.537719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:26.537800 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:26.537905 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:26.538656 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.539428 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.540878 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:26.541327 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:26.541457 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:26.545665 (+ 4208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:40771: BlockingRecv error: recv got EOF from 127.5.71.129:40771 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 23 20:47:27 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:27 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:27.016121 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:27.005947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:39721)
0923 20:47:27.006501 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:27.006519 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:27.006779 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:27.007222 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:27.007227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:27.007666 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:27.007900 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:27.007911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.009084 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.009091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:27.010170 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:27.010183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.010994 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.011001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:27.011210 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:27.011947 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:27.011967 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:27.015755 (+ 3788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":316,"threads_started":1}
W20260923 20:47:27.016422 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:27.006050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:39721 (local address 127.5.71.190:45569)
0923 20:47:27.006371 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:27.006378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:27.006622 (+ 244us) server_negotiation.cc:407] Connection header received
0923 20:47:27.006980 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:27.006984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:27.007051 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:27.007194 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:27.008062 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.008934 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.010339 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.010811 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.010980 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:27.016122 (+ 5142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:39721: BlockingRecv error: recv got EOF from 127.5.71.130:39721 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Sep 23 20:47:27 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:27 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:27.509207 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:27.499573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:54493)
0923 20:47:27.500278 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:27.500303 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:27.500433 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:27.500813 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:27.500818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:27.501343 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:27.501527 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:27.501539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.502941 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.502946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:27.504008 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:27.504023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.504874 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.504879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:27.505006 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:27.505775 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:27.505797 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:27.508863 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":422,"threads_started":1}
W20260923 20:47:27.509434 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:27.499886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:54493 (local address 127.5.71.190:45569)
0923 20:47:27.500111 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:27.500117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:27.500423 (+ 306us) server_negotiation.cc:407] Connection header received
0923 20:47:27.500590 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:27.500594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:27.500657 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:27.500736 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:27.501735 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.502779 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.504173 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.504655 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.504761 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:27.509100 (+ 4339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:54493: BlockingRecv error: recv got EOF from 127.5.71.131:54493 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 23 20:47:27 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:27 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:27.556581 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:27.547087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:34489)
0923 20:47:27.547683 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:27.547698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:27.547901 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:27.548324 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:27.548329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:27.548882 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:27.549075 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:27.549085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.550120 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.550128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:27.551196 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:27.551209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.552133 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.552140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:27.552314 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:27.553224 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:27.553244 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:27.556182 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":316,"threads_started":1}
W20260923 20:47:27.556664 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:27.547250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:34489 (local address 127.5.71.190:45569)
0923 20:47:27.547527 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:27.547533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:27.547825 (+ 292us) server_negotiation.cc:407] Connection header received
0923 20:47:27.548046 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:27.548050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:27.548145 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:27.548299 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:27.549240 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.549979 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.551349 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:27.551971 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:27.552115 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:27.556410 (+ 4295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:34489: BlockingRecv error: recv got EOF from 127.5.71.129:34489 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 23 20:47:28 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:28 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:28.026084 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:28.017116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:33801)
0923 20:47:28.017654 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:28.017670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:28.017804 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:28.018224 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:28.018228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:28.018692 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:28.018891 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:28.018902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.020068 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.020072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:28.021073 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:28.021086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.021768 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.021772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:28.021887 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:28.022579 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:28.022597 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:28.025677 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":310,"threads_started":1}
W20260923 20:47:28.026214 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:28.017311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33801 (local address 127.5.71.190:45569)
0923 20:47:28.017492 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:28.017497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:28.017786 (+ 289us) server_negotiation.cc:407] Connection header received
0923 20:47:28.017964 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:28.017968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:28.018033 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:28.018131 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:28.019038 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.019888 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.021235 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.021635 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.021744 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:28.025935 (+ 4191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:33801: BlockingRecv error: recv got EOF from 127.5.71.130:33801 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 23 20:47:28 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:28 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:28.528157 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:28.510418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:55465)
0923 20:47:28.511050 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:28.511071 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:28.511274 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:28.511655 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:28.511660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:28.512313 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:28.512481 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:28.512490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.513724 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.513733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:28.515442 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:28.515459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.516372 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.516379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:28.516563 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:28.524190 (+ 7627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:28.524217 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:28.527731 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":382,"threads_started":1}
W20260923 20:47:28.528368 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:28.510815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:55465 (local address 127.5.71.190:45569)
0923 20:47:28.511026 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:28.511033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:28.511191 (+ 158us) server_negotiation.cc:407] Connection header received
0923 20:47:28.511433 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:28.511440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:28.511508 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:28.511618 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:28.512633 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.513522 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.515588 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.516212 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.516351 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:28.528105 (+ 11754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:55465: BlockingRecv error: recv got EOF from 127.5.71.131:55465 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 23 20:47:28 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:28 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:28.567926 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:28.557896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:52695)
0923 20:47:28.558551 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:28.558569 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:28.558692 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:28.559112 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:28.559117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:28.559748 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:28.559952 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:28.559962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.561259 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.561263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:28.562223 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:28.562235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.563227 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.563231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:28.563391 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:28.564173 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:28.564192 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:28.567591 (+ 3399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":395,"threads_started":1}
W20260923 20:47:28.568189 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:28.557892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:52695 (local address 127.5.71.190:45569)
0923 20:47:28.558102 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:28.558107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:28.558682 (+ 575us) server_negotiation.cc:407] Connection header received
0923 20:47:28.558836 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:28.558840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:28.558919 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:28.559016 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:28.560137 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.561072 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.562406 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:28.563062 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:28.563192 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:28.567908 (+ 4716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:52695: BlockingRecv error: recv got EOF from 127.5.71.129:52695 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 23 20:47:29 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:29 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:29.035943 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:29.027119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:57629)
0923 20:47:29.027724 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:29.027746 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:29.027936 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:29.028332 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:29.028337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:29.028768 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:29.028940 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:29.028950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.029986 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.029992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:29.030918 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:29.030929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.031688 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.031693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:29.031914 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:29.032563 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:29.032581 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:29.035529 (+ 2948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":316,"threads_started":1}
W20260923 20:47:29.036094 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:29.027111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:57629 (local address 127.5.71.190:45569)
0923 20:47:29.027357 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:29.027366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:29.027862 (+ 496us) server_negotiation.cc:407] Connection header received
0923 20:47:29.028112 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:29.028115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:29.028179 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:29.028286 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:29.029119 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.029826 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.031064 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.031526 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.031670 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:29.035826 (+ 4156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:57629: BlockingRecv error: recv got EOF from 127.5.71.130:57629 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 23 20:47:29 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:29 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:29.538815 14712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:29.529295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:38681)
0923 20:47:29.529976 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:29.529993 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:29.530127 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:29.530523 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:29.530527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:29.530914 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:29.531069 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:29.531077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.532572 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.532576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:29.533665 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:29.533677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.534524 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.534530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:29.534704 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:29.535378 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:29.535396 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:29.538482 (+ 3086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":432,"threads_started":1}
W20260923 20:47:29.538959 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:29.529398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:38681 (local address 127.5.71.190:45569)
0923 20:47:29.529634 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:29.529639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:29.530111 (+ 472us) server_negotiation.cc:407] Connection header received
0923 20:47:29.530253 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:29.530257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:29.530327 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:29.530427 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:29.531232 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.532398 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.533859 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.534368 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.534503 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:29.538710 (+ 4207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:38681: BlockingRecv error: recv got EOF from 127.5.71.131:38681 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 23 20:47:29 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:29 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:29.578449 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:29.569093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:34257)
0923 20:47:29.569696 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:29.569712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:29.569876 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:29.570292 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:29.570297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:29.570712 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:29.570879 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:29.570888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.571991 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.571997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:29.572959 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:29.572970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.573769 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.573777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:29.573938 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:29.574657 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:29.574711 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:29.578162 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":309,"threads_started":1}
W20260923 20:47:29.578756 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:29.569134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:34257 (local address 127.5.71.190:45569)
0923 20:47:29.569378 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:29.569384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:29.569816 (+ 432us) server_negotiation.cc:407] Connection header received
0923 20:47:29.570017 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:29.570022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:29.570117 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:29.570267 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:29.571029 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.571855 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.573100 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:29.573549 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:29.573697 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:29.578477 (+ 4780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:34257: BlockingRecv error: recv got EOF from 127.5.71.129:34257 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 23 20:47:30 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:30 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:30.047139 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:30.037152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:47117)
0923 20:47:30.037910 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:30.037935 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:30.038100 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:30.038554 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:30.038558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:30.038933 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:30.039110 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:30.039118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.040527 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.040533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:30.042026 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:30.042039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.042806 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.042811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:30.042943 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:30.043657 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:30.043676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:30.046869 (+ 3193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":409,"threads_started":1}
W20260923 20:47:30.047420 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:30.037439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:47117 (local address 127.5.71.190:45569)
0923 20:47:30.037638 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:30.037645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:30.038053 (+ 408us) server_negotiation.cc:407] Connection header received
0923 20:47:30.038317 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:30.038322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:30.038410 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:30.038512 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:30.039283 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.040322 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.042210 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.042628 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.042723 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:30.047157 (+ 4434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:47117: BlockingRecv error: recv got EOF from 127.5.71.130:47117 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 23 20:47:30 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:30 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:30.549149 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:30.539766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:33707)
0923 20:47:30.540464 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:30.540480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:30.540645 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:30.540979 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:30.540984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:30.541393 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:30.541556 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:30.541566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.542719 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.542725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:30.543652 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:30.543663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.544432 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.544439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:30.544608 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:30.545255 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:30.545276 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:30.548888 (+ 3612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":318,"threads_started":1}
W20260923 20:47:30.549384 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:30.540085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:33707 (local address 127.5.71.190:45569)
0923 20:47:30.540331 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:30.540339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:30.540564 (+ 225us) server_negotiation.cc:407] Connection header received
0923 20:47:30.540767 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:30.540771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:30.540844 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:30.540952 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:30.541775 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.542575 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.543830 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.544250 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.544393 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:30.549117 (+ 4724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:33707: BlockingRecv error: recv got EOF from 127.5.71.131:33707 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 23 20:47:30 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:30 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:30.589306 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:30.579546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:46643)
0923 20:47:30.580194 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:30.580217 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:30.580406 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:30.580820 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:30.580825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:30.581291 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:30.581475 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:30.581484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.582650 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.582659 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:30.583884 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:30.583899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.584842 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.584849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:30.585039 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:30.585792 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:30.585811 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:30.589068 (+ 3257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":352,"threads_started":1}
W20260923 20:47:30.589658 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:30.579782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:46643 (local address 127.5.71.190:45569)
0923 20:47:30.580195 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:30.580204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:30.580323 (+ 119us) server_negotiation.cc:407] Connection header received
0923 20:47:30.580562 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:30.580566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:30.580639 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:30.580761 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:30.581647 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.582493 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.584040 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:30.584673 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:30.584821 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:30.589355 (+ 4534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:46643: BlockingRecv error: recv got EOF from 127.5.71.129:46643 (error 108)
Metrics: {"server-negotiator.queue_time_us":217}
Sep 23 20:47:31 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:31 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:31.057101 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:31.048258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:55359)
0923 20:47:31.048835 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:31.048853 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:31.049048 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:31.049413 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:31.049418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:31.049831 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:31.050006 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:31.050014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.051082 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.051087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:31.052044 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:31.052054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.052795 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.052801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:31.052991 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:31.053630 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:31.053647 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:31.056751 (+ 3104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":330,"threads_started":1}
W20260923 20:47:31.057209 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:31.048468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:55359 (local address 127.5.71.190:45569)
0923 20:47:31.048713 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:31.048719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:31.048960 (+ 241us) server_negotiation.cc:407] Connection header received
0923 20:47:31.049183 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:31.049195 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:31.049266 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:31.049376 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:31.050164 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.050935 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.052189 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.052628 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.052764 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:31.056996 (+ 4232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:55359: BlockingRecv error: recv got EOF from 127.5.71.130:55359 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 23 20:47:31 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:31 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.131@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:31.559576 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:31.550279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.131:36349)
0923 20:47:31.550870 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:31.550889 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:31.551030 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:31.551363 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:31.551366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:31.551934 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:31.552213 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:31.552231 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.553530 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.553538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:31.554694 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:31.554707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.555370 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.555375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:31.555497 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:31.556148 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:31.556167 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:31.559293 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":344,"threads_started":1}
W20260923 20:47:31.559854 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:31.550436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:36349 (local address 127.5.71.190:45569)
0923 20:47:31.550614 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:31.550618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:31.550969 (+ 351us) server_negotiation.cc:407] Connection header received
0923 20:47:31.551139 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:31.551143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:31.551206 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:31.551303 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:31.552417 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.553369 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.554835 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.555239 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.555347 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:31.559533 (+ 4186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.131:36349: BlockingRecv error: recv got EOF from 127.5.71.131:36349 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 20:47:31 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:31 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.129@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:31.599901 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:31.590641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.129:54897)
0923 20:47:31.591230 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:31.591252 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:31.591411 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:31.591932 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:31.591937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:31.592427 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:31.592707 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:31.592721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.593911 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.593916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:31.594874 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:31.594886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.595624 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.595628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:31.595767 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:31.596525 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:31.596542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:31.599502 (+ 2960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":309,"threads_started":1}
W20260923 20:47:31.599984 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:31.590716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:54897 (local address 127.5.71.190:45569)
0923 20:47:31.590960 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:31.590969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:31.591352 (+ 383us) server_negotiation.cc:407] Connection header received
0923 20:47:31.591569 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:31.591573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:31.591635 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:31.591907 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:31.592926 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.593766 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.595032 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:31.595479 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:31.595579 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:31.599715 (+ 4136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.129:54897: BlockingRecv error: recv got EOF from 127.5.71.129:54897 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Sep 23 20:47:32 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 20:47:32 dist-test-slave-dn29 krb5kdc[13673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.71.130@KRBTEST.COM for oryx/127.5.71.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 20:47:32.068325 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:32.058405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45569 (local address 127.5.71.130:51133)
0923 20:47:32.059031 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:32.059047 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:32.059153 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:32.059692 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:32.059696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:32.060160 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:32.060355 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:32.060365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.061583 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.061587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:32.062614 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:32.062625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.063397 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.063402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:32.063549 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:32.064436 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:32.064467 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:32.068001 (+ 3534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.71.190:45569: Server oryx/127.5.71.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":408,"threads_started":1}
W20260923 20:47:32.068512 14411 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:32.058475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:51133 (local address 127.5.71.190:45569)
0923 20:47:32.058659 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:32.058664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:32.059125 (+ 461us) server_negotiation.cc:407] Connection header received
0923 20:47:32.059323 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:32.059328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:32.059413 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:32.059526 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:32.060616 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.061432 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.062766 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.063236 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.063343 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:32.068255 (+ 4912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.71.130:51133: BlockingRecv error: recv got EOF from 127.5.71.130:51133 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:47:32.357918 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14262
I20260923 20:47:32.372810 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:32.359254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38950 (local address 127.5.71.129:40927)
0923 20:47:32.359991 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:32.360000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:32.360028 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:47:32.360101 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:32.360108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:32.360412 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:32.360591 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:32.362191 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.363884 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.365907 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.366511 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.366631 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:32.369630 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:32.369641 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:32.369657 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:32.369695 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:32.372318 (+ 2623us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0923 20:47:32.372586 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38950: Request ticket server kudu/127.5.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":420,"threads_started":1}
W20260923 20:47:32.372942 14722 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38950: Request ticket server kudu/127.5.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260923 20:47:32.373137 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14262: Not authorized: Client connection negotiation failed: client connection to 127.5.71.129:40927: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.71.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260923 20:47:32.373420 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14262
I20260923 20:47:32.392154 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14413
I20260923 20:47:32.405975 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:32.393434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.5.71.130:46221)
0923 20:47:32.394080 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:32.394087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:32.394116 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:47:32.394181 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:32.394188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:32.394535 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:32.394716 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:32.396153 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.397289 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.398730 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.399279 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.399376 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:32.402236 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:32.402246 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:32.402263 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:32.402301 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:32.405356 (+ 3055us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0923 20:47:32.405658 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35536: Request ticket server kudu/127.5.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":351,"threads_started":1}
W20260923 20:47:32.406095 14723 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35536: Request ticket server kudu/127.5.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260923 20:47:32.406183 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14413: Not authorized: Client connection negotiation failed: client connection to 127.5.71.130:46221: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.71.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260923 20:47:32.406358 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14413
I20260923 20:47:32.423221 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14544
I20260923 20:47:32.437007 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:32.424443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.5.71.131:43687)
0923 20:47:32.425258 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:32.425266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:32.425296 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:47:32.425369 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:32.425375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:32.425652 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:32.425819 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:32.427260 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.428670 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.429913 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:32.430567 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:32.430700 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:32.433449 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:32.433458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:32.433475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:32.433513 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:32.436331 (+ 2818us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0923 20:47:32.436729 (+ 398us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45798: Request ticket server kudu/127.5.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":520,"threads_started":1}
W20260923 20:47:32.437166 14724 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45798: Request ticket server kudu/127.5.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260923 20:47:32.437256 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14544: Not authorized: Client connection negotiation failed: client connection to 127.5.71.131:43687: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.5.71.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260923 20:47:32.437422 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14544
I20260923 20:47:32.453980 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14193
I20260923 20:47:32.455479 14255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:32.578751 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14193
2026-09-23T20:47:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21398 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T20:47:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:32Z Disabled control of system clock
I20260923 20:47:32.658447 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:33311
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 20:47:33.024114 14729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:33.024462 14729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:33.024649 14729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:33.033305 14729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:33.033438 14729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:33.033551 14729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:33.033648 14729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:33.033733 14729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:33.043557 14729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:33311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:33.045028 14729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:33.046943 14729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:33.060443 14737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:33.060452 14735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:33.060362 14734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:33.060869 14729 server_base.cc:1034] running on GCE node
I20260923 20:47:33.061741 14729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:33.063227 14729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:33.064453 14729 hybrid_clock.cc:648] HybridClock initialized: now 1790196453064410 us; error 29 us; skew 500 ppm
I20260923 20:47:33.064839 14729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:33.067413 14729 webserver.cc:460] Webserver started at http://127.5.71.190:45389/ using document root <none> and password file <none>
I20260923 20:47:33.068218 14729 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:33.068323 14729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:33.068598 14729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:33.071724 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "a61fcd56c99243b291ad68c9a494cae0"
format_stamp: "Formatted at 2026-09-23 20:47:33 on dist-test-slave-dn29"
server_key: "cef972b135faa1ae81bc0b08dde651ba"
server_key_iv: "403eb1fffb4d46d8f20c23722a58d647"
server_key_version: "encryptionkey@0"
I20260923 20:47:33.072785 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "a61fcd56c99243b291ad68c9a494cae0"
format_stamp: "Formatted at 2026-09-23 20:47:33 on dist-test-slave-dn29"
server_key: "cef972b135faa1ae81bc0b08dde651ba"
server_key_iv: "403eb1fffb4d46d8f20c23722a58d647"
server_key_version: "encryptionkey@0"
I20260923 20:47:33.078469 14729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:47:33.082461 14743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:33.084311 14729 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:47:33.084477 14729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "a61fcd56c99243b291ad68c9a494cae0"
format_stamp: "Formatted at 2026-09-23 20:47:33 on dist-test-slave-dn29"
server_key: "cef972b135faa1ae81bc0b08dde651ba"
server_key_iv: "403eb1fffb4d46d8f20c23722a58d647"
server_key_version: "encryptionkey@0"
I20260923 20:47:33.084669 14729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:33.108634 14729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:33.109490 14729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:33.109736 14729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:33.129206 14729 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:33311
I20260923 20:47:33.129252 14794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:33311 every 8 connection(s)
I20260923 20:47:33.130640 14729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:47:33.133960 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14729
I20260923 20:47:33.134155 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:47:33.134888 5406 external_mini_cluster.cc:1428] Setting key e4d3589b1fd08b84ab962122f7cc7b90
I20260923 20:47:33.135113 14795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:33.152379 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:33.137204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.5.71.190:33311)
0923 20:47:33.138474 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:33.138498 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:33.138555 (+ 57us) server_negotiation.cc:407] Connection header received
0923 20:47:33.140891 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:33.140928 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:33.141674 (+ 746us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:33.142227 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:33.142879 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:33.145182 (+ 2303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:33.146505 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:33.147637 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:33.147878 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:33.149186 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:33.149198 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:33.149221 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:33.150081 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:33.150201 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:33.150236 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:33.150363 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:33.150553 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 20:47:33.150928 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":394,"threads_started":1}
I20260923 20:47:33.152717 14795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0: Bootstrap starting.
I20260923 20:47:33.156809 14795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:33.157938 14795 log.cc:824] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:33.161586 14795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0: No bootstrap required, opened a new log
I20260923 20:47:33.169516 14795 raft_consensus.cc:348] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61fcd56c99243b291ad68c9a494cae0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 33311 } }
I20260923 20:47:33.169927 14795 raft_consensus.cc:374] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:33.170040 14795 raft_consensus.cc:729] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a61fcd56c99243b291ad68c9a494cae0, State: Initialized, Role: FOLLOWER
I20260923 20:47:33.170719 14795 consensus_queue.cc:260] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61fcd56c99243b291ad68c9a494cae0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 33311 } }
I20260923 20:47:33.170928 14795 raft_consensus.cc:388] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:47:33.171073 14795 raft_consensus.cc:482] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:47:33.171271 14795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:33.173836 14795 raft_consensus.cc:504] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61fcd56c99243b291ad68c9a494cae0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 33311 } }
I20260923 20:47:33.174467 14795 leader_election.cc:302] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a61fcd56c99243b291ad68c9a494cae0; no voters:
I20260923 20:47:33.175112 14795 leader_election.cc:288] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:47:33.175323 14800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:33.176218 14800 raft_consensus.cc:686] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [term 1 LEADER]: Becoming Leader. State: Replica: a61fcd56c99243b291ad68c9a494cae0, State: Running, Role: LEADER
I20260923 20:47:33.176769 14800 consensus_queue.cc:237] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61fcd56c99243b291ad68c9a494cae0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 33311 } }
I20260923 20:47:33.177556 14795 sys_catalog.cc:564] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:47:33.179761 14802 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a61fcd56c99243b291ad68c9a494cae0. Latest consensus state: current_term: 1 leader_uuid: "a61fcd56c99243b291ad68c9a494cae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61fcd56c99243b291ad68c9a494cae0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 33311 } } }
I20260923 20:47:33.180431 14802 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:33.180049 14801 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a61fcd56c99243b291ad68c9a494cae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61fcd56c99243b291ad68c9a494cae0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 33311 } } }
I20260923 20:47:33.181257 14801 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:33.181502 14809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:33.187741 14809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:33.197223 14809 catalog_manager.cc:1347] Generated new cluster ID: 979c801380dc470191d4e96f7be691cd
I20260923 20:47:33.197409 14809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:33.219298 14809 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:47:33.221349 14809 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:33.234541 14809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a61fcd56c99243b291ad68c9a494cae0: Generated new TSK 0
I20260923 20:47:33.235461 14809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:47:33.247035 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:33311
--builtin_ntp_servers=127.5.71.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 20:47:33.598165 14820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:33.598448 14820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:33.598593 14820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:33.606118 14820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:33.606236 14820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:33.606407 14820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:47:33.616753 14820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:33311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:33.618144 14820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:33.620150 14820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:33.634035 14825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:33.634053 14828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:33.634341 14826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:33.637876 14820 server_base.cc:1034] running on GCE node
I20260923 20:47:33.638728 14820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:33.640059 14820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:33.641337 14820 hybrid_clock.cc:648] HybridClock initialized: now 1790196453641258 us; error 71 us; skew 500 ppm
I20260923 20:47:33.641749 14820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:33.655925 14820 webserver.cc:460] Webserver started at http://127.5.71.129:46797/ using document root <none> and password file <none>
I20260923 20:47:33.656855 14820 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:33.657008 14820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:33.657404 14820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:33.660737 14820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "9f5493ae6b8c4ca7b4afacca35e909c9"
format_stamp: "Formatted at 2026-09-23 20:47:33 on dist-test-slave-dn29"
server_key: "6d6ae54a0293297499256a18aff93dcc"
server_key_iv: "9843ae340c815d9533776b375223f667"
server_key_version: "encryptionkey@0"
I20260923 20:47:33.661645 14820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "9f5493ae6b8c4ca7b4afacca35e909c9"
format_stamp: "Formatted at 2026-09-23 20:47:33 on dist-test-slave-dn29"
server_key: "6d6ae54a0293297499256a18aff93dcc"
server_key_iv: "9843ae340c815d9533776b375223f667"
server_key_version: "encryptionkey@0"
I20260923 20:47:33.667459 14820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260923 20:47:33.671500 14834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:33.673259 14820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:47:33.673476 14820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "9f5493ae6b8c4ca7b4afacca35e909c9"
format_stamp: "Formatted at 2026-09-23 20:47:33 on dist-test-slave-dn29"
server_key: "6d6ae54a0293297499256a18aff93dcc"
server_key_iv: "9843ae340c815d9533776b375223f667"
server_key_version: "encryptionkey@0"
I20260923 20:47:33.673863 14820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:33.694525 14820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:33.695436 14820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:33.695760 14820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:33.696959 14820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:33.698490 14820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:33.698619 14820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:33.698762 14820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:33.698853 14820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:33.733433 14820 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:35699
I20260923 20:47:33.733486 14946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:35699 every 8 connection(s)
I20260923 20:47:33.735064 14820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:47:33.744004 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14820
I20260923 20:47:33.744203 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:47:33.744894 5406 external_mini_cluster.cc:1428] Setting key 4740cf6028b9035eb30f403285d317e6
I20260923 20:47:33.747622 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:33311
--builtin_ntp_servers=127.5.71.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 20:47:33.748351 14949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:33.737798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:43997 (local address 127.5.71.190:33311)
0923 20:47:33.738271 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:33.738276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:33.739511 (+ 1235us) server_negotiation.cc:407] Connection header received
0923 20:47:33.741345 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:33.741350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:33.741440 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:33.741562 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:33.742672 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:33.743379 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:33.745261 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:33.745828 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:33.745989 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:33.747463 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:33.747468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:33.747473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:33.747500 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:33.747544 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:33.747556 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:33.747656 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:33.747928 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 20:47:33.748021 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":247,"threads_started":1}
I20260923 20:47:33.749385 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:33.738314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:33311 (local address 127.5.71.129:43997)
0923 20:47:33.739256 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:33.739299 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:33.741062 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:33.741804 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:33.741813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:33.741925 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:33.742505 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:33.742521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:33.743557 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:33.743567 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:33.745082 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:33.745098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:33.746103 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:33.746109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:33.746342 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:33.747120 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:47:33.747122 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:33.747164 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:47:33.747272 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:33.747662 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:33.747675 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:33.747684 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 20:47:33.747922 (+ 238us) client_negotiation.cc:240] Negotiation successful
0923 20:47:33.748346 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":279,"threads_started":1}
I20260923 20:47:33.750985 14947 heartbeater.cc:347] Connected to a master server at 127.5.71.190:33311
I20260923 20:47:33.751295 14947 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:33.752148 14947 heartbeater.cc:511] Master 127.5.71.190:33311 requested a full tablet report, sending...
I20260923 20:47:33.754647 14760 ts_manager.cc:194] Registered new tserver with Master: 9f5493ae6b8c4ca7b4afacca35e909c9 (127.5.71.129:35699)
I20260923 20:47:33.756403 14760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.129:43997
W20260923 20:47:34.080735 14951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:34.081080 14951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:34.081243 14951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:34.089062 14951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:34.089215 14951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:34.089386 14951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:47:34.099388 14951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:33311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:34.100841 14951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:34.102722 14951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:34.116284 14959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:34.116365 14956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:34.116367 14957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:34.118098 14951 server_base.cc:1034] running on GCE node
I20260923 20:47:34.118748 14951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:34.119741 14951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:34.121018 14951 hybrid_clock.cc:648] HybridClock initialized: now 1790196454120962 us; error 48 us; skew 500 ppm
I20260923 20:47:34.121358 14951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:34.123749 14951 webserver.cc:460] Webserver started at http://127.5.71.130:43007/ using document root <none> and password file <none>
I20260923 20:47:34.124554 14951 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:34.124688 14951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:34.125020 14951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:34.128070 14951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "2bbb9c8fb41e4651aa9b78681093a8bf"
format_stamp: "Formatted at 2026-09-23 20:47:34 on dist-test-slave-dn29"
server_key: "900b636cb1cc3954908e874e6344ebc4"
server_key_iv: "6413f9e04c3b71bd7f91360f70005094"
server_key_version: "encryptionkey@0"
I20260923 20:47:34.129011 14951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "2bbb9c8fb41e4651aa9b78681093a8bf"
format_stamp: "Formatted at 2026-09-23 20:47:34 on dist-test-slave-dn29"
server_key: "900b636cb1cc3954908e874e6344ebc4"
server_key_iv: "6413f9e04c3b71bd7f91360f70005094"
server_key_version: "encryptionkey@0"
I20260923 20:47:34.134544 14951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:47:34.138242 14965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:34.139695 14951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260923 20:47:34.139910 14951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "2bbb9c8fb41e4651aa9b78681093a8bf"
format_stamp: "Formatted at 2026-09-23 20:47:34 on dist-test-slave-dn29"
server_key: "900b636cb1cc3954908e874e6344ebc4"
server_key_iv: "6413f9e04c3b71bd7f91360f70005094"
server_key_version: "encryptionkey@0"
I20260923 20:47:34.140136 14951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:34.163333 14951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:34.164314 14951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:34.164626 14951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:34.165637 14951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:34.167378 14951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:34.167507 14951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:34.167654 14951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:34.167757 14951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:34.203955 14951 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:43511
I20260923 20:47:34.204041 15077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:43511 every 8 connection(s)
I20260923 20:47:34.205299 14951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:47:34.212881 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 14951
I20260923 20:47:34.213059 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:47:34.213495 5406 external_mini_cluster.cc:1428] Setting key ba2149469be6137ebaa4ad64496ec1ee
I20260923 20:47:34.216248 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:33311
--builtin_ntp_servers=127.5.71.148:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 20:47:34.218873 14949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.208372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:33343 (local address 127.5.71.190:33311)
0923 20:47:34.208578 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:34.208582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:34.210025 (+ 1443us) server_negotiation.cc:407] Connection header received
0923 20:47:34.212060 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:34.212066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:34.212162 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:34.212286 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:34.213522 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.214328 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.215937 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.216404 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.216540 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.218056 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:34.218061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:34.218065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:34.218083 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:34.218114 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:34.218126 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:34.218238 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:34.218478 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 20:47:34.218572 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:47:34.220018 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.208784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:33311 (local address 127.5.71.130:33343)
0923 20:47:34.209828 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:34.209876 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:34.211694 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:34.212530 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:34.212539 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:34.212637 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:34.213349 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:34.213367 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.214490 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.214497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:34.215746 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:34.215757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.216631 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.216639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:34.216880 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.217618 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:47:34.217621 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:34.217739 (+ 118us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:47:34.217839 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:34.218235 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:34.218247 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:34.218257 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 20:47:34.218467 (+ 210us) client_negotiation.cc:240] Negotiation successful
0923 20:47:34.218888 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":284,"threads_started":1}
I20260923 20:47:34.221590 15078 heartbeater.cc:347] Connected to a master server at 127.5.71.190:33311
I20260923 20:47:34.221936 15078 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:34.222798 15078 heartbeater.cc:511] Master 127.5.71.190:33311 requested a full tablet report, sending...
I20260923 20:47:34.224398 14760 ts_manager.cc:194] Registered new tserver with Master: 2bbb9c8fb41e4651aa9b78681093a8bf (127.5.71.130:43511)
I20260923 20:47:34.225617 14760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.130:33343
W20260923 20:47:34.567186 15081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:34.567518 15081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:34.567682 15081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:34.575591 15081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:34.575723 15081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:34.575945 15081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:34.586200 15081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:33311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:34.587669 15081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:34.589854 15081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:34.605631 15089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:34.605751 15087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:34.605962 15086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:34.606585 15081 server_base.cc:1034] running on GCE node
I20260923 20:47:34.607273 15081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:34.608371 15081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:34.609637 15081 hybrid_clock.cc:648] HybridClock initialized: now 1790196454609573 us; error 53 us; skew 500 ppm
I20260923 20:47:34.610119 15081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:34.612787 15081 webserver.cc:460] Webserver started at http://127.5.71.131:36235/ using document root <none> and password file <none>
I20260923 20:47:34.613525 15081 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:34.613677 15081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:34.614022 15081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:34.617147 15081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "336ed55bb7134a9fb7a280d56a0e0114"
format_stamp: "Formatted at 2026-09-23 20:47:34 on dist-test-slave-dn29"
server_key: "d35c7a908fdc82bfdd063759d894f089"
server_key_iv: "2c5d07691d99e5da247dad8a1e451811"
server_key_version: "encryptionkey@0"
I20260923 20:47:34.617990 15081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "336ed55bb7134a9fb7a280d56a0e0114"
format_stamp: "Formatted at 2026-09-23 20:47:34 on dist-test-slave-dn29"
server_key: "d35c7a908fdc82bfdd063759d894f089"
server_key_iv: "2c5d07691d99e5da247dad8a1e451811"
server_key_version: "encryptionkey@0"
I20260923 20:47:34.623790 15081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:47:34.627972 15095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:34.629639 15081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:47:34.629825 15081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "336ed55bb7134a9fb7a280d56a0e0114"
format_stamp: "Formatted at 2026-09-23 20:47:34 on dist-test-slave-dn29"
server_key: "d35c7a908fdc82bfdd063759d894f089"
server_key_iv: "2c5d07691d99e5da247dad8a1e451811"
server_key_version: "encryptionkey@0"
I20260923 20:47:34.630056 15081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:34.662899 15081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:34.664014 15081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:34.664366 15081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:34.665506 15081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:34.667157 15081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:34.667295 15081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:34.667443 15081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:34.667538 15081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:34.706425 15081 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:38637
I20260923 20:47:34.706523 15207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:38637 every 8 connection(s)
I20260923 20:47:34.708117 15081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:34.712364 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15081
I20260923 20:47:34.712550 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:47:34.713114 5406 external_mini_cluster.cc:1428] Setting key f97650baa5f6a895f72c1d73f2bedaa3
I20260923 20:47:34.722891 14949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.711100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:37665 (local address 127.5.71.190:33311)
0923 20:47:34.711347 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:34.711354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:34.713099 (+ 1745us) server_negotiation.cc:407] Connection header received
0923 20:47:34.715267 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:34.715273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:34.715397 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:34.715509 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:34.716877 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.717632 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.719603 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.720200 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.720321 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.721940 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:34.721946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:34.721949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:34.721979 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:34.722011 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:34.722024 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:34.722110 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:34.722413 (+ 303us) server_negotiation.cc:299] Negotiation successful
0923 20:47:34.722517 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:47:34.723954 15210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.711818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:33311 (local address 127.5.71.131:37665)
0923 20:47:34.712901 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:34.712948 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:34.715004 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:34.715736 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:34.715745 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:34.715911 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:34.716702 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:34.716722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.718029 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.718033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:34.719435 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:34.719449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.720479 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.720484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:34.720704 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.721574 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:47:34.721576 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:34.721626 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:47:34.721720 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:34.722142 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:34.722152 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:34.722159 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 20:47:34.722347 (+ 188us) client_negotiation.cc:240] Negotiation successful
0923 20:47:34.722677 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":343,"threads_started":1}
I20260923 20:47:34.725523 15208 heartbeater.cc:347] Connected to a master server at 127.5.71.190:33311
I20260923 20:47:34.725891 15208 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:34.726754 15208 heartbeater.cc:511] Master 127.5.71.190:33311 requested a full tablet report, sending...
I20260923 20:47:34.728648 14760 ts_manager.cc:194] Registered new tserver with Master: 336ed55bb7134a9fb7a280d56a0e0114 (127.5.71.131:38637)
I20260923 20:47:34.729585 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:47:34.729846 14760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.131:37665
W20260923 20:47:34.737087 14949 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:34.735188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.5.71.190:33311)
0923 20:47:34.735410 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:34.735418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:34.736041 (+ 623us) server_negotiation.cc:407] Connection header received
0923 20:47:34.736231 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:34.736234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:34.736308 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:34.736392 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:34.736872 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42124: BlockingRecv error: recv got EOF from 127.0.0.1:42124 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 20:47:34.737980 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14820
I20260923 20:47:34.747674 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.739136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.5.71.129:35699)
0923 20:47:34.739817 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:34.739828 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:34.740206 (+ 378us) server_negotiation.cc:407] Connection header received
0923 20:47:34.740293 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:34.740303 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:34.740589 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:34.740767 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:34.741244 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.742462 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.743922 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.744671 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.744800 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.746101 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:34.746113 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:34.746129 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:34.746818 (+ 689us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:34.746873 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:34.746897 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:34.747037 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:34.747193 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 20:47:34.747327 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":358,"threads_started":1}
I20260923 20:47:34.749166 14942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:34.835709 14947 heartbeater.cc:503] Master 127.5.71.190:33311 was elected leader, sending a full tablet report...
I20260923 20:47:34.837013 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14820
W20260923 20:47:34.853924 14746 connection.cc:535] server connection from 127.5.71.129:43997 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 20:47:34.854274 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14951
I20260923 20:47:34.862358 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.855162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.5.71.130:43511)
0923 20:47:34.855689 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:34.855696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:34.855724 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:47:34.855823 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:34.855838 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:34.856102 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:34.856289 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:34.856720 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.857765 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.859008 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.859609 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.859753 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.860874 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:34.860882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:34.860898 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:34.861506 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:34.861552 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:34.861574 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:34.861702 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:34.861883 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 20:47:34.862012 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":256,"threads_started":1}
I20260923 20:47:34.863667 15073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:34.950229 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14951
I20260923 20:47:34.966791 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15081
I20260923 20:47:34.976423 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:34.967927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40298 (local address 127.5.71.131:38637)
0923 20:47:34.968651 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:34.968661 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:34.968706 (+ 45us) server_negotiation.cc:407] Connection header received
0923 20:47:34.968774 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:34.968781 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:34.969018 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:34.969174 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:34.969684 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.970810 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.972228 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:34.972843 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:34.972954 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:34.974315 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:34.974324 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:34.974340 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:34.975018 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:34.975067 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:34.975087 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:34.975192 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:34.975441 (+ 249us) server_negotiation.cc:299] Negotiation successful
0923 20:47:34.975608 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":368,"threads_started":1}
I20260923 20:47:34.978142 15203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:35.065807 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15081
I20260923 20:47:35.082435 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 14729
I20260923 20:47:35.084044 14790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:35.187299 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 14729
2026-09-23T20:47:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2604 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T20:47:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:35Z Disabled control of system clock
I20260923 20:47:35.263708 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:37259
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:37259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 20:47:35.602120 15227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:35.602420 15227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:35.602504 15227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:35.610234 15227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:35.610361 15227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:35.610400 15227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:35.610450 15227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:35.610495 15227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:35.620608 15227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:37259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:37259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:35.621958 15227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:35.623690 15227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:35.636873 15232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:35.636900 15235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:35.637015 15233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:35.637876 15227 server_base.cc:1034] running on GCE node
I20260923 20:47:35.638808 15227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:35.640462 15227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:35.641840 15227 hybrid_clock.cc:648] HybridClock initialized: now 1790196455641769 us; error 55 us; skew 500 ppm
I20260923 20:47:35.642266 15227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:35.645035 15227 webserver.cc:460] Webserver started at http://127.5.71.190:35495/ using document root <none> and password file <none>
I20260923 20:47:35.645807 15227 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:35.645951 15227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:35.646287 15227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:35.649467 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "9b15e3fc27004722b6287c7418dff94f"
format_stamp: "Formatted at 2026-09-23 20:47:35 on dist-test-slave-dn29"
server_key: "f0915aa3fbd880c1adf06c260d512137"
server_key_iv: "75d3761cecd4d126d273abc6d67c7b48"
server_key_version: "encryptionkey@0"
I20260923 20:47:35.650328 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "9b15e3fc27004722b6287c7418dff94f"
format_stamp: "Formatted at 2026-09-23 20:47:35 on dist-test-slave-dn29"
server_key: "f0915aa3fbd880c1adf06c260d512137"
server_key_iv: "75d3761cecd4d126d273abc6d67c7b48"
server_key_version: "encryptionkey@0"
I20260923 20:47:35.655884 15227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260923 20:47:35.660028 15241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:35.661736 15227 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:47:35.661990 15227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "9b15e3fc27004722b6287c7418dff94f"
format_stamp: "Formatted at 2026-09-23 20:47:35 on dist-test-slave-dn29"
server_key: "f0915aa3fbd880c1adf06c260d512137"
server_key_iv: "75d3761cecd4d126d273abc6d67c7b48"
server_key_version: "encryptionkey@0"
I20260923 20:47:35.662261 15227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:35.669020 15227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:35.669898 15227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:35.670228 15227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:35.690215 15227 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:37259
I20260923 20:47:35.690260 15292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:37259 every 8 connection(s)
I20260923 20:47:35.691699 15227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:47:35.695650 15293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:35.698475 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15227
I20260923 20:47:35.698691 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:47:35.699460 5406 external_mini_cluster.cc:1428] Setting key dabb7089d1f2aaeb87da460c277b0b1d
I20260923 20:47:35.708577 15293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f: Bootstrap starting.
I20260923 20:47:35.708952 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:35.701755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.5.71.190:37259)
0923 20:47:35.702979 (+ 1224us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:35.702991 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:35.703036 (+ 45us) server_negotiation.cc:407] Connection header received
0923 20:47:35.704910 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:35.704934 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:35.705424 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:35.705878 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:35.707085 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:35.707091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:35.707105 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:35.707121 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:35.707182 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:35.707199 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:35.707455 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:35.707601 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 20:47:35.707925 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":379,"threads_started":1}
I20260923 20:47:35.712359 15293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:35.713483 15293 log.cc:824] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:35.716991 15293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f: No bootstrap required, opened a new log
I20260923 20:47:35.724852 15293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b15e3fc27004722b6287c7418dff94f" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37259 } }
I20260923 20:47:35.725286 15293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:35.725404 15293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b15e3fc27004722b6287c7418dff94f, State: Initialized, Role: FOLLOWER
I20260923 20:47:35.726166 15293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b15e3fc27004722b6287c7418dff94f" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37259 } }
I20260923 20:47:35.726469 15293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:47:35.726681 15293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:47:35.726863 15293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:35.729162 15293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b15e3fc27004722b6287c7418dff94f" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37259 } }
I20260923 20:47:35.729693 15293 leader_election.cc:302] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b15e3fc27004722b6287c7418dff94f; no voters:
I20260923 20:47:35.730325 15293 leader_election.cc:288] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:47:35.730497 15298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:35.731195 15298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [term 1 LEADER]: Becoming Leader. State: Replica: 9b15e3fc27004722b6287c7418dff94f, State: Running, Role: LEADER
I20260923 20:47:35.731992 15298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b15e3fc27004722b6287c7418dff94f" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37259 } }
I20260923 20:47:35.732774 15293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:47:35.734784 15299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b15e3fc27004722b6287c7418dff94f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b15e3fc27004722b6287c7418dff94f" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37259 } } }
I20260923 20:47:35.735036 15300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b15e3fc27004722b6287c7418dff94f. Latest consensus state: current_term: 1 leader_uuid: "9b15e3fc27004722b6287c7418dff94f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b15e3fc27004722b6287c7418dff94f" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37259 } } }
I20260923 20:47:35.735260 15299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:35.735349 15300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:35.736217 15306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:35.742326 15306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:35.753957 15306 catalog_manager.cc:1347] Generated new cluster ID: d9d536335cc0491e957975cd12816b2e
I20260923 20:47:35.754096 15306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:35.782204 15306 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:47:35.782706 15306 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:35.795501 15306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b15e3fc27004722b6287c7418dff94f: Generated new TSK 0
I20260923 20:47:35.796669 15306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:47:35.802775 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37259
--builtin_ntp_servers=127.5.71.148:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 20:47:36.158381 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:36.158762 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:36.158937 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:36.166639 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:36.166747 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:36.166914 15317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:47:36.176936 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:36.178380 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:36.180595 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:36.195232 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:36.195206 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:36.195217 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:36.196372 15317 server_base.cc:1034] running on GCE node
I20260923 20:47:36.197009 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:36.198029 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:36.199275 15317 hybrid_clock.cc:648] HybridClock initialized: now 1790196456199222 us; error 48 us; skew 500 ppm
I20260923 20:47:36.199651 15317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:36.202188 15317 webserver.cc:460] Webserver started at http://127.5.71.129:33575/ using document root <none> and password file <none>
I20260923 20:47:36.202908 15317 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:36.203042 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:36.203392 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:36.208667 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "2b76f97a8246453f9c314653743398ce"
format_stamp: "Formatted at 2026-09-23 20:47:36 on dist-test-slave-dn29"
server_key: "7e5d94e64b0623173a7c357910c331a8"
server_key_iv: "0f867a598dcbedc9762de55def9388b7"
server_key_version: "encryptionkey@0"
I20260923 20:47:36.209529 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "2b76f97a8246453f9c314653743398ce"
format_stamp: "Formatted at 2026-09-23 20:47:36 on dist-test-slave-dn29"
server_key: "7e5d94e64b0623173a7c357910c331a8"
server_key_iv: "0f867a598dcbedc9762de55def9388b7"
server_key_version: "encryptionkey@0"
I20260923 20:47:36.215143 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:47:36.218865 15331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:36.220398 15317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:47:36.220592 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "2b76f97a8246453f9c314653743398ce"
format_stamp: "Formatted at 2026-09-23 20:47:36 on dist-test-slave-dn29"
server_key: "7e5d94e64b0623173a7c357910c331a8"
server_key_iv: "0f867a598dcbedc9762de55def9388b7"
server_key_version: "encryptionkey@0"
I20260923 20:47:36.220834 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:36.226783 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:36.227612 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:36.227950 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:36.229048 15317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:36.230513 15317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:36.230641 15317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:36.230784 15317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:36.230875 15317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:36.266378 15317 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:37627
I20260923 20:47:36.266347 15443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:37627 every 8 connection(s)
I20260923 20:47:36.268028 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:47:36.277251 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:36.270929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:41851 (local address 127.5.71.190:37259)
0923 20:47:36.271445 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:36.271465 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:36.272718 (+ 1253us) server_negotiation.cc:407] Connection header received
0923 20:47:36.274583 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:36.274588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:36.274645 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:36.274751 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:36.276355 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:36.276367 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:36.276370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:36.276390 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:36.276449 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:36.276461 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:36.276575 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:36.276943 (+ 368us) server_negotiation.cc:299] Negotiation successful
0923 20:47:36.277005 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":238,"threads_started":1}
I20260923 20:47:36.277952 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15317
I20260923 20:47:36.278174 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:47:36.278402 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:36.271407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:37259 (local address 127.5.71.129:41851)
0923 20:47:36.272536 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:36.272575 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:36.274286 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:36.275026 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:36.275039 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:36.275151 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:36.276026 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:47:36.276028 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:36.276070 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:47:36.276183 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:36.276590 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:36.276600 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:36.276608 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 20:47:36.276950 (+ 342us) client_negotiation.cc:240] Negotiation successful
0923 20:47:36.277192 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":403,"threads_started":1}
I20260923 20:47:36.278846 5406 external_mini_cluster.cc:1428] Setting key 5477becc612c093d10561f533ae91b82
I20260923 20:47:36.280339 15444 heartbeater.cc:347] Connected to a master server at 127.5.71.190:37259
I20260923 20:47:36.280745 15444 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:36.281399 15444 heartbeater.cc:511] Master 127.5.71.190:37259 requested a full tablet report, sending...
I20260923 20:47:36.281742 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37259
--builtin_ntp_servers=127.5.71.148:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 20:47:36.283861 15258 ts_manager.cc:194] Registered new tserver with Master: 2b76f97a8246453f9c314653743398ce (127.5.71.129:37627)
W20260923 20:47:36.622215 15448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:36.622535 15448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:36.622699 15448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:36.630205 15448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:36.630321 15448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:36.630493 15448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:47:36.640610 15448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:36.641999 15448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:36.643832 15448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:36.657781 15453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:36.657749 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:36.657823 15454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:36.664327 15448 server_base.cc:1034] running on GCE node
I20260923 20:47:36.665139 15448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:36.666301 15448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:36.667544 15448 hybrid_clock.cc:648] HybridClock initialized: now 1790196456667452 us; error 84 us; skew 500 ppm
I20260923 20:47:36.667979 15448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:36.670660 15448 webserver.cc:460] Webserver started at http://127.5.71.130:39459/ using document root <none> and password file <none>
I20260923 20:47:36.671510 15448 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:36.671665 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:36.672118 15448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:36.675501 15448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "af14c9899d8647bc8159a001378be01e"
format_stamp: "Formatted at 2026-09-23 20:47:36 on dist-test-slave-dn29"
server_key: "0dc1457a72f246fab495e311e5215bf6"
server_key_iv: "f62ca6836f69bc45c99d4a0f9168a478"
server_key_version: "encryptionkey@0"
I20260923 20:47:36.676460 15448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "af14c9899d8647bc8159a001378be01e"
format_stamp: "Formatted at 2026-09-23 20:47:36 on dist-test-slave-dn29"
server_key: "0dc1457a72f246fab495e311e5215bf6"
server_key_iv: "f62ca6836f69bc45c99d4a0f9168a478"
server_key_version: "encryptionkey@0"
I20260923 20:47:36.682302 15448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260923 20:47:36.686408 15462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:36.688107 15448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:47:36.688261 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "af14c9899d8647bc8159a001378be01e"
format_stamp: "Formatted at 2026-09-23 20:47:36 on dist-test-slave-dn29"
server_key: "0dc1457a72f246fab495e311e5215bf6"
server_key_iv: "f62ca6836f69bc45c99d4a0f9168a478"
server_key_version: "encryptionkey@0"
I20260923 20:47:36.688470 15448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:36.694773 15448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:36.695633 15448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:36.695999 15448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:36.697026 15448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:36.698725 15448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:36.698825 15448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:36.698993 15448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:36.699096 15448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:36.739432 15448 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:40217
I20260923 20:47:36.739535 15574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:40217 every 8 connection(s)
I20260923 20:47:36.740880 15448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:47:36.747488 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15448
I20260923 20:47:36.747726 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:47:36.748229 5406 external_mini_cluster.cc:1428] Setting key 27eb6f5058d86cd09ebfc93bcf0b71dc
I20260923 20:47:36.750264 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:36.743581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:54133 (local address 127.5.71.190:37259)
0923 20:47:36.743932 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:36.743939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:36.745641 (+ 1702us) server_negotiation.cc:407] Connection header received
0923 20:47:36.747691 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:36.747697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:36.747765 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:36.747928 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:36.749480 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:36.749485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:36.749489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:36.749505 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:36.749535 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:36.749547 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:36.749689 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:36.749896 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 20:47:36.749966 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:47:36.751250 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37259
--builtin_ntp_servers=127.5.71.148:34667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 20:47:36.751309 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:36.744320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:37259 (local address 127.5.71.130:54133)
0923 20:47:36.745441 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:36.745486 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:36.747290 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:36.748243 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:36.748253 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:36.748406 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:36.749134 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:47:36.749137 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:36.749184 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:47:36.749270 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:36.749651 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:36.749662 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:36.749671 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 20:47:36.749987 (+ 316us) client_negotiation.cc:240] Negotiation successful
0923 20:47:36.750286 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":404,"threads_started":1}
I20260923 20:47:36.752801 15575 heartbeater.cc:347] Connected to a master server at 127.5.71.190:37259
I20260923 20:47:36.753150 15575 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:36.753913 15575 heartbeater.cc:511] Master 127.5.71.190:37259 requested a full tablet report, sending...
I20260923 20:47:36.755640 15258 ts_manager.cc:194] Registered new tserver with Master: af14c9899d8647bc8159a001378be01e (127.5.71.130:40217)
W20260923 20:47:37.098872 15578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:37.099189 15578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:37.099332 15578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:37.107071 15578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:37.107190 15578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:37.107372 15578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:37.116786 15578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:37.118191 15578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:37.120110 15578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:37.134400 15583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:37.134305 15584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:37.134262 15586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:37.135670 15578 server_base.cc:1034] running on GCE node
I20260923 20:47:37.136437 15578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:37.137557 15578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:37.138814 15578 hybrid_clock.cc:648] HybridClock initialized: now 1790196457138717 us; error 42 us; skew 500 ppm
I20260923 20:47:37.139176 15578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:47:37.142000 15578 webserver.cc:460] Webserver started at http://127.5.71.131:40741/ using document root <none> and password file <none>
I20260923 20:47:37.142810 15578 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:37.142908 15578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:37.143334 15578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:37.146584 15578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "33fa27bc2e334db08d99c33665021dfa"
format_stamp: "Formatted at 2026-09-23 20:47:37 on dist-test-slave-dn29"
server_key: "806a0b134ae14c4ff82c152898143603"
server_key_iv: "836b8cd56812a7e692e71d18711a29eb"
server_key_version: "encryptionkey@0"
I20260923 20:47:37.147611 15578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "33fa27bc2e334db08d99c33665021dfa"
format_stamp: "Formatted at 2026-09-23 20:47:37 on dist-test-slave-dn29"
server_key: "806a0b134ae14c4ff82c152898143603"
server_key_iv: "836b8cd56812a7e692e71d18711a29eb"
server_key_version: "encryptionkey@0"
I20260923 20:47:37.154045 15578 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:47:37.158008 15592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:37.159597 15578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260923 20:47:37.159864 15578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "33fa27bc2e334db08d99c33665021dfa"
format_stamp: "Formatted at 2026-09-23 20:47:37 on dist-test-slave-dn29"
server_key: "806a0b134ae14c4ff82c152898143603"
server_key_iv: "836b8cd56812a7e692e71d18711a29eb"
server_key_version: "encryptionkey@0"
I20260923 20:47:37.160113 15578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:37.166049 15578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:37.166901 15578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:37.167225 15578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:37.168329 15578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:37.169782 15578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:37.169907 15578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:37.170047 15578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:37.170141 15578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:37.207356 15578 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:35663
I20260923 20:47:37.207463 15704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:35663 every 8 connection(s)
I20260923 20:47:37.208839 15578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:37.218472 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15578
I20260923 20:47:37.218597 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:37.211945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:46249 (local address 127.5.71.190:37259)
0923 20:47:37.212149 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:37.212155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:37.213929 (+ 1774us) server_negotiation.cc:407] Connection header received
0923 20:47:37.215844 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:37.215851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:37.215930 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:37.216082 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:37.217819 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:37.217824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:37.217827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:37.217845 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:37.217879 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:37.217891 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:37.218000 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:37.218267 (+ 267us) server_negotiation.cc:299] Negotiation successful
0923 20:47:37.218325 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:47:37.218691 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:47:37.219280 5406 external_mini_cluster.cc:1428] Setting key aa40213960cb6665d2063f02b23e1c29
I20260923 20:47:37.220137 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:37.212538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:37259 (local address 127.5.71.131:46249)
0923 20:47:37.213684 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:37.213724 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:37.215512 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:37.216500 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:37.216510 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:37.216655 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:37.217467 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:47:37.217469 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:37.217513 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:47:37.217620 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:37.217991 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:37.218023 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:37.218034 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 20:47:37.218396 (+ 362us) client_negotiation.cc:240] Negotiation successful
0923 20:47:37.218727 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":381,"threads_started":1}
I20260923 20:47:37.221910 15705 heartbeater.cc:347] Connected to a master server at 127.5.71.190:37259
I20260923 20:47:37.222273 15705 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:37.222956 15705 heartbeater.cc:511] Master 127.5.71.190:37259 requested a full tablet report, sending...
I20260923 20:47:37.224651 15258 ts_manager.cc:194] Registered new tserver with Master: 33fa27bc2e334db08d99c33665021dfa (127.5.71.131:35663)
I20260923 20:47:37.235481 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 20:47:37.241847 15446 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:47:37.240226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.5.71.190:37259)
0923 20:47:37.240425 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:37.240432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:37.240810 (+ 378us) server_negotiation.cc:407] Connection header received
0923 20:47:37.241025 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:37.241037 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:37.241092 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:37.241199 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:37.241553 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44242: BlockingRecv error: recv got EOF from 127.0.0.1:44242 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 20:47:37.243021 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15317
I20260923 20:47:37.246786 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:37.244003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.5.71.129:37627)
0923 20:47:37.244545 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:37.244560 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:37.244587 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:47:37.244645 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:37.244652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:37.244876 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:37.245009 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:37.245963 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:37.245970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:37.245982 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:37.245999 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:37.246051 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:37.246066 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:37.246173 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:37.246391 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 20:47:37.246480 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":257,"threads_started":1}
I20260923 20:47:37.248302 15439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:37.329553 15444 heartbeater.cc:503] Master 127.5.71.190:37259 was elected leader, sending a full tablet report...
I20260923 20:47:37.331282 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15317
W20260923 20:47:37.344539 15243 connection.cc:535] server connection from 127.5.71.129:41851 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 20:47:37.344668 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15448
I20260923 20:47:37.348800 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:37.345695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.5.71.130:40217)
0923 20:47:37.346296 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:37.346304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:37.346333 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:47:37.346402 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:37.346409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:37.346675 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:37.346865 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:37.348012 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:37.348020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:37.348033 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:37.348046 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:37.348094 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:37.348108 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:37.348272 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:37.348382 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 20:47:37.348500 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":283,"threads_started":1}
I20260923 20:47:37.350211 15570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:37.435588 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15448
I20260923 20:47:37.452319 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15578
I20260923 20:47:37.456437 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:37.453329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.5.71.131:35663)
0923 20:47:37.454046 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:37.454060 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:37.454102 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:47:37.454168 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:37.454175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:37.454406 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:37.454537 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:37.455551 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:37.455557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:37.455570 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:47:37.455581 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:37.455621 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:47:37.455638 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:37.455775 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:37.456066 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 20:47:37.456158 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":314,"threads_started":1}
I20260923 20:47:37.457759 15700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:37.542068 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15578
I20260923 20:47:37.558082 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15227
I20260923 20:47:37.559725 15288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:37.662766 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15227
2026-09-23T20:47:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2475 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:47:37 dist-test-slave-dn29 krb5kdc[15723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:47:37 dist-test-slave-dn29 krb5kdc[15723](info): set up 2 sockets
Sep 23 20:47:37 dist-test-slave-dn29 krb5kdc[15723](info): commencing operation
krb5kdc: starting...
W20260923 20:47:38.789830 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:47:38 dist-test-slave-dn29 krb5kdc[15723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:47:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:47:39.040678 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:34487
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:34487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:39.384608 15736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:39.384934 15736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:39.385025 15736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:39.393623 15736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:47:39.393806 15736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:39.393903 15736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:39.394037 15736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:47:39.394100 15736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:47:39.404592 15736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:34487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:39.406224 15736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:39.407506 15736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:39.420089 15744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:39.420137 15741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:39.420112 15742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:39.420907 15736 server_base.cc:1034] running on GCE node
I20260923 20:47:39.421895 15736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:39.423404 15736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:39.424640 15736 hybrid_clock.cc:648] HybridClock initialized: now 1790196459424548 us; error 83 us; skew 500 ppm
I20260923 20:47:39.425017 15736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:39 dist-test-slave-dn29 krb5kdc[15723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196459, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:39.438344 15736 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:47:39.440305 15736 webserver.cc:460] Webserver started at http://127.5.71.190:39927/ using document root <none> and password file <none>
I20260923 20:47:39.441113 15736 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:39.441290 15736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:39.441656 15736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:39.445226 15736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "c77f50316dc54befa93fc58162597cb0"
format_stamp: "Formatted at 2026-09-23 20:47:39 on dist-test-slave-dn29"
server_key: "6c32674b91dc6db0214d3d1a646d307b"
server_key_iv: "8048cf39dc85370431eaf20b48fb216f"
server_key_version: "encryptionkey@0"
I20260923 20:47:39.446089 15736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "c77f50316dc54befa93fc58162597cb0"
format_stamp: "Formatted at 2026-09-23 20:47:39 on dist-test-slave-dn29"
server_key: "6c32674b91dc6db0214d3d1a646d307b"
server_key_iv: "8048cf39dc85370431eaf20b48fb216f"
server_key_version: "encryptionkey@0"
I20260923 20:47:39.452176 15736 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:47:39.456199 15751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:39.457683 15736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:47:39.457993 15736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "c77f50316dc54befa93fc58162597cb0"
format_stamp: "Formatted at 2026-09-23 20:47:39 on dist-test-slave-dn29"
server_key: "6c32674b91dc6db0214d3d1a646d307b"
server_key_iv: "8048cf39dc85370431eaf20b48fb216f"
server_key_version: "encryptionkey@0"
I20260923 20:47:39.458217 15736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:39.478760 15736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:39.483783 15736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:39.484143 15736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:39.506088 15736 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:34487
I20260923 20:47:39.506108 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:34487 every 8 connection(s)
I20260923 20:47:39.507630 15736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:47:39.511766 15803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:39.516363 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15736
I20260923 20:47:39.516608 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:47:39.517264 5406 external_mini_cluster.cc:1428] Setting key 46184d61bbf6479a0b6717304e471a51
I20260923 20:47:39.524837 15803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0: Bootstrap starting.
I20260923 20:47:39.528733 15803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:39.529892 15803 log.cc:824] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:39.533979 15803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0: No bootstrap required, opened a new log
Sep 23 20:47:39 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:39.543296 15803 raft_consensus.cc:348] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77f50316dc54befa93fc58162597cb0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34487 } }
I20260923 20:47:39.543762 15803 raft_consensus.cc:374] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:39.543955 15803 raft_consensus.cc:729] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c77f50316dc54befa93fc58162597cb0, State: Initialized, Role: FOLLOWER
I20260923 20:47:39.544504 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:39.519425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.5.71.190:34487)
0923 20:47:39.520306 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:39.520325 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:39.520384 (+ 59us) server_negotiation.cc:407] Connection header received
0923 20:47:39.523037 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:39.523074 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:39.523633 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:39.524196 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:39.525882 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:39.527961 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:39.529606 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:39.530710 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:39.530851 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:39.536260 (+ 5409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:39.536269 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:39.536286 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:39.536335 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:39.539866 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:39.540635 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:39.540653 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:39.540660 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:39.540840 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:39.541247 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:39.541262 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:39.541264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:39.541864 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:39.542215 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:39.542672 (+ 457us) server_negotiation.cc:299] Negotiation successful
0923 20:47:39.543112 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":380,"threads_started":1}
I20260923 20:47:39.544677 15803 consensus_queue.cc:260] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77f50316dc54befa93fc58162597cb0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34487 } }
I20260923 20:47:39.544991 15803 raft_consensus.cc:388] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:47:39.545181 15803 raft_consensus.cc:482] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:47:39.545392 15803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:39.548830 15803 raft_consensus.cc:504] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77f50316dc54befa93fc58162597cb0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34487 } }
I20260923 20:47:39.549460 15803 leader_election.cc:302] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c77f50316dc54befa93fc58162597cb0; no voters:
I20260923 20:47:39.550240 15803 leader_election.cc:288] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:47:39.550307 15808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:39.551118 15808 raft_consensus.cc:686] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [term 1 LEADER]: Becoming Leader. State: Replica: c77f50316dc54befa93fc58162597cb0, State: Running, Role: LEADER
I20260923 20:47:39.551725 15808 consensus_queue.cc:237] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77f50316dc54befa93fc58162597cb0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34487 } }
I20260923 20:47:39.552706 15803 sys_catalog.cc:564] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:47:39.554715 15809 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c77f50316dc54befa93fc58162597cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77f50316dc54befa93fc58162597cb0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34487 } } }
I20260923 20:47:39.555266 15809 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:39.554828 15810 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c77f50316dc54befa93fc58162597cb0. Latest consensus state: current_term: 1 leader_uuid: "c77f50316dc54befa93fc58162597cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c77f50316dc54befa93fc58162597cb0" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 34487 } } }
I20260923 20:47:39.555469 15810 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0 [sys.catalog]: This master's current role is: LEADER
I20260923 20:47:39.557188 15813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:47:39.563007 15813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:47:39.572628 15813 catalog_manager.cc:1347] Generated new cluster ID: 8de4242a50434a0fba18f54a76a8b405
I20260923 20:47:39.572774 15813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:47:39.618017 15813 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:47:39.619551 15813 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:47:39.629981 15813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0: Generated new TSK 0
I20260923 20:47:39.630926 15813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:47:39.708832 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34487
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:40.051779 15829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:40.052145 15829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:40.052250 15829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:40.060032 15829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:40.060148 15829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:40.060267 15829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:47:40.070649 15829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:40.072140 15829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:40.073493 15829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:40.087595 15834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:40.087628 15835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:40.088092 15837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:40.088658 15829 server_base.cc:1034] running on GCE node
I20260923 20:47:40.089351 15829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:40.090372 15829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:40.091634 15829 hybrid_clock.cc:648] HybridClock initialized: now 1790196460091582 us; error 44 us; skew 500 ppm
I20260923 20:47:40.092036 15829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:40 dist-test-slave-dn29 krb5kdc[15723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196460, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:40.095407 15829 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:47:40.097285 15829 webserver.cc:460] Webserver started at http://127.5.71.129:41357/ using document root <none> and password file <none>
I20260923 20:47:40.098024 15829 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:40.098176 15829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:40.098491 15829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:40.101537 15829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "bf1ddd2af45c4696ade550f9cdcdea11"
format_stamp: "Formatted at 2026-09-23 20:47:40 on dist-test-slave-dn29"
server_key: "1d071d3fe5ce4af1519ae56057b51420"
server_key_iv: "4195cb373370002d7a59f4063b1ff5d7"
server_key_version: "encryptionkey@0"
I20260923 20:47:40.102380 15829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "bf1ddd2af45c4696ade550f9cdcdea11"
format_stamp: "Formatted at 2026-09-23 20:47:40 on dist-test-slave-dn29"
server_key: "1d071d3fe5ce4af1519ae56057b51420"
server_key_iv: "4195cb373370002d7a59f4063b1ff5d7"
server_key_version: "encryptionkey@0"
I20260923 20:47:40.108059 15829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260923 20:47:40.112300 15844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:40.114028 15829 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260923 20:47:40.114279 15829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "bf1ddd2af45c4696ade550f9cdcdea11"
format_stamp: "Formatted at 2026-09-23 20:47:40 on dist-test-slave-dn29"
server_key: "1d071d3fe5ce4af1519ae56057b51420"
server_key_iv: "4195cb373370002d7a59f4063b1ff5d7"
server_key_version: "encryptionkey@0"
I20260923 20:47:40.114537 15829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:40.159089 15829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:40.164265 15829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:40.164605 15829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:40.165695 15829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:40.167246 15829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:40.167369 15829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:40.167541 15829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:40.167624 15829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:40.206060 15829 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:39853
I20260923 20:47:40.206311 15956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:39853 every 8 connection(s)
I20260923 20:47:40.207479 15829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:47:40.214221 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15829
I20260923 20:47:40.214473 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:47:40.215106 5406 external_mini_cluster.cc:1428] Setting key 372d3715cfe460db7bb0cf4a7d9f3e0a
Sep 23 20:47:40 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196460, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:40.230031 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:40.210146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:42421 (local address 127.5.71.190:34487)
0923 20:47:40.210595 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:40.210602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:40.211625 (+ 1023us) server_negotiation.cc:407] Connection header received
0923 20:47:40.213544 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:40.213549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:40.213619 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:40.213856 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:40.215710 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.216496 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.218189 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.218960 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.219101 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:40.223292 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:40.223301 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:40.223308 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:40.223353 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:40.226459 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:40.227468 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:40.227472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:40.227474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:40.227594 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:40.228179 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:40.228182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:40.228185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:40.228473 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:40.228594 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:40.229561 (+ 967us) server_negotiation.cc:299] Negotiation successful
0923 20:47:40.229763 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":205,"threads_started":1}
I20260923 20:47:40.231635 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:40.210719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:34487 (local address 127.5.71.129:42421)
0923 20:47:40.211452 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:40.211497 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:40.213264 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:40.214021 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:40.214031 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:40.214730 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:40.215488 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:40.215523 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.216667 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.216672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:40.218008 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:40.218019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.219155 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.219160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:40.219502 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:40.220245 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:40.220268 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:40.223017 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:40.226609 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:40.226622 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:40.226645 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:40.227340 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:40.227714 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:40.227721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:40.227728 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:40.228013 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:40.228617 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:40.228629 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:40.229128 (+ 499us) client_negotiation.cc:769] Sending connection context
0923 20:47:40.229438 (+ 310us) client_negotiation.cc:240] Negotiation successful
0923 20:47:40.230073 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":302,"threads_started":1}
I20260923 20:47:40.233446 15957 heartbeater.cc:347] Connected to a master server at 127.5.71.190:34487
I20260923 20:47:40.233865 15957 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:40.234735 15957 heartbeater.cc:511] Master 127.5.71.190:34487 requested a full tablet report, sending...
I20260923 20:47:40.237171 15768 ts_manager.cc:194] Registered new tserver with Master: bf1ddd2af45c4696ade550f9cdcdea11 (127.5.71.129:39853)
I20260923 20:47:40.239032 15768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:42421
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:47:40.273913 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34487
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:40.624495 15963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:40.624856 15963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:40.624960 15963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:40.632647 15963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:40.632755 15963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:40.632869 15963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:47:40.642731 15963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:40.644104 15963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:40.645401 15963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:40.659422 15968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:40.659492 15971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:40.659425 15969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:40.660147 15963 server_base.cc:1034] running on GCE node
I20260923 20:47:40.660856 15963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:40.661993 15963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:40.663255 15963 hybrid_clock.cc:648] HybridClock initialized: now 1790196460663202 us; error 46 us; skew 500 ppm
I20260923 20:47:40.663623 15963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:40 dist-test-slave-dn29 krb5kdc[15723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196460, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:40.667249 15963 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:47:40.668998 15963 webserver.cc:460] Webserver started at http://127.5.71.130:40207/ using document root <none> and password file <none>
I20260923 20:47:40.669739 15963 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:40.669867 15963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:40.670190 15963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:40.673313 15963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "04bf90afbc2b44c88163b2c3cb7f783f"
format_stamp: "Formatted at 2026-09-23 20:47:40 on dist-test-slave-dn29"
server_key: "30e684d61b638849643cf93c2bee8a1a"
server_key_iv: "562fec5c50b6314311a9af84c8a51e39"
server_key_version: "encryptionkey@0"
I20260923 20:47:40.674134 15963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "04bf90afbc2b44c88163b2c3cb7f783f"
format_stamp: "Formatted at 2026-09-23 20:47:40 on dist-test-slave-dn29"
server_key: "30e684d61b638849643cf93c2bee8a1a"
server_key_iv: "562fec5c50b6314311a9af84c8a51e39"
server_key_version: "encryptionkey@0"
I20260923 20:47:40.679867 15963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260923 20:47:40.683768 15978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:40.685751 15963 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:47:40.685972 15963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "04bf90afbc2b44c88163b2c3cb7f783f"
format_stamp: "Formatted at 2026-09-23 20:47:40 on dist-test-slave-dn29"
server_key: "30e684d61b638849643cf93c2bee8a1a"
server_key_iv: "562fec5c50b6314311a9af84c8a51e39"
server_key_version: "encryptionkey@0"
I20260923 20:47:40.686216 15963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:40.712311 15963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:40.716885 15963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:40.717211 15963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:40.718326 15963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:40.719985 15963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:40.720080 15963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:40.720237 15963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:40.720307 15963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:40.755048 15963 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:34783
I20260923 20:47:40.755079 16090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:34783 every 8 connection(s)
I20260923 20:47:40.756400 15963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:47:40.759155 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 15963
I20260923 20:47:40.759315 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:47:40.759884 5406 external_mini_cluster.cc:1428] Setting key 1accaefc3149a2634e16d31601c4a030
Sep 23 20:47:40 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196460, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:40.780293 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:40.759250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:32963 (local address 127.5.71.190:34487)
0923 20:47:40.759835 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:40.759841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:40.760983 (+ 1142us) server_negotiation.cc:407] Connection header received
0923 20:47:40.763074 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:40.763082 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:40.763182 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:40.763300 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:40.764995 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.766120 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.767756 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.768552 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.768697 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:40.773107 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:40.773116 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:40.773124 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:40.773171 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:40.776746 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:40.777528 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:40.777536 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:40.777543 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:40.777710 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:40.778157 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:40.778165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:40.778171 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:40.778579 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:40.778745 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:40.779477 (+ 732us) server_negotiation.cc:299] Negotiation successful
0923 20:47:40.779760 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
I20260923 20:47:40.781008 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:40.759954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:34487 (local address 127.5.71.130:32963)
0923 20:47:40.760815 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:40.760872 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:40.762760 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:40.763510 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:40.763521 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:40.764228 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:40.764825 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:40.764843 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.766332 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.766336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:40.767570 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:40.767582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:40.768823 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:40.768828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:40.769132 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:40.769933 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:40.769967 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:40.772846 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:40.776911 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:40.776919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:40.776935 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:40.777384 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:40.777833 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:40.777836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:40.777839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:40.778025 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:40.778735 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:40.778741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:40.779067 (+ 326us) client_negotiation.cc:769] Sending connection context
0923 20:47:40.779324 (+ 257us) client_negotiation.cc:240] Negotiation successful
0923 20:47:40.779757 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":348,"threads_started":1}
I20260923 20:47:40.782583 16091 heartbeater.cc:347] Connected to a master server at 127.5.71.190:34487
I20260923 20:47:40.782974 16091 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:40.784004 16091 heartbeater.cc:511] Master 127.5.71.190:34487 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:47:40.785787 15768 ts_manager.cc:194] Registered new tserver with Master: 04bf90afbc2b44c88163b2c3cb7f783f (127.5.71.130:34783)
I20260923 20:47:40.786638 15768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:32963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:47:40.819444 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34487
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:47:41.157213 16097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:47:41.157531 16097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:47:41.157717 16097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:47:41.165786 16097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:47:41.165927 16097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:47:41.166136 16097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:47:41.176246 16097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:34487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:47:41.177709 16097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:47:41.179139 16097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:47:41.194048 16102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:41.193977 16105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:47:41.193953 16103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:47:41.194551 16097 server_base.cc:1034] running on GCE node
I20260923 20:47:41.195560 16097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:47:41.196671 16097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:47:41.197901 16097 hybrid_clock.cc:648] HybridClock initialized: now 1790196461197845 us; error 40 us; skew 500 ppm
I20260923 20:47:41.198318 16097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:47:41 dist-test-slave-dn29 krb5kdc[15723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196461, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:47:41.202216 16097 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:47:41.204056 16097 webserver.cc:460] Webserver started at http://127.5.71.131:37407/ using document root <none> and password file <none>
I20260923 20:47:41.204843 16097 fs_manager.cc:359] Metadata directory not provided
I20260923 20:47:41.204954 16097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:47:41.205359 16097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:47:41.209764 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58"
format_stamp: "Formatted at 2026-09-23 20:47:41 on dist-test-slave-dn29"
server_key: "900a87a904da7a6eddbca368889779ba"
server_key_iv: "e03107879c41887036ca7e9e0ce54c3a"
server_key_version: "encryptionkey@0"
I20260923 20:47:41.210811 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58"
format_stamp: "Formatted at 2026-09-23 20:47:41 on dist-test-slave-dn29"
server_key: "900a87a904da7a6eddbca368889779ba"
server_key_iv: "e03107879c41887036ca7e9e0ce54c3a"
server_key_version: "encryptionkey@0"
I20260923 20:47:41.217092 16097 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260923 20:47:41.221431 16112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:41.223099 16097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260923 20:47:41.223304 16097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58"
format_stamp: "Formatted at 2026-09-23 20:47:41 on dist-test-slave-dn29"
server_key: "900a87a904da7a6eddbca368889779ba"
server_key_iv: "e03107879c41887036ca7e9e0ce54c3a"
server_key_version: "encryptionkey@0"
I20260923 20:47:41.223518 16097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:47:41.242381 15957 heartbeater.cc:503] Master 127.5.71.190:34487 was elected leader, sending a full tablet report...
I20260923 20:47:41.248317 16097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:47:41.253949 16097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:47:41.254285 16097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:47:41.255406 16097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:47:41.256927 16097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:47:41.257007 16097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:41.257143 16097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:47:41.257205 16097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:47:41.295234 16097 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:34779
I20260923 20:47:41.295351 16224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:34779 every 8 connection(s)
I20260923 20:47:41.296713 16097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:47:41.304018 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 16097
I20260923 20:47:41.304284 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:47:41.304757 5406 external_mini_cluster.cc:1428] Setting key ba20ad832ef05044f7968942a2bd5390
Sep 23 20:47:41 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196461, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:47:41.317648 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.299377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:36007 (local address 127.5.71.190:34487)
0923 20:47:41.299971 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.299976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.300857 (+ 881us) server_negotiation.cc:407] Connection header received
0923 20:47:41.302630 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.302636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.302728 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.302852 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:41.304601 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.305582 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.307309 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.307948 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.308089 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.311665 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:41.311671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:41.311674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:41.311718 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:41.314513 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.315292 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.315296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.315299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.315418 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.315920 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.315924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.315926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.316260 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:41.316386 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.317160 (+ 774us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.317359 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":250,"threads_started":1}
I20260923 20:47:41.318845 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.299914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:34487 (local address 127.5.71.131:36007)
0923 20:47:41.300635 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:41.300679 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:41.302354 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:41.303085 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:41.303093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:41.303632 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:47:41.304420 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.304445 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.305727 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.305732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.307163 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.307177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.308156 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.308162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.308364 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.309142 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:47:41.309166 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:47:41.311448 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:47:41.314648 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:41.314655 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:41.314670 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:41.315181 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:41.315514 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:47:41.315517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:47:41.315519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:47:41.315692 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:47:41.316373 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:47:41.316379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:47:41.316703 (+ 324us) client_negotiation.cc:769] Sending connection context
0923 20:47:41.317008 (+ 305us) client_negotiation.cc:240] Negotiation successful
0923 20:47:41.317402 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":291,"threads_started":1}
I20260923 20:47:41.320652 16225 heartbeater.cc:347] Connected to a master server at 127.5.71.190:34487
I20260923 20:47:41.321126 16225 heartbeater.cc:464] Registering TS with master...
I20260923 20:47:41.321889 16225 heartbeater.cc:511] Master 127.5.71.190:34487 requested a full tablet report, sending...
I20260923 20:47:41.323441 15768 ts_manager.cc:194] Registered new tserver with Master: e4fb9e4c7cf042e6b4e12c08fa193c58 (127.5.71.131:34779)
I20260923 20:47:41.324370 15768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:36007
I20260923 20:47:41.333801 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:47:41.353008 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.338684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.5.71.190:34487)
0923 20:47:41.338950 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.338957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.339405 (+ 448us) server_negotiation.cc:407] Connection header received
0923 20:47:41.339633 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.339638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.339725 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.339862 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:41.341552 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.342357 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.343701 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.344239 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.344386 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.346997 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:41.347002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:41.347005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:41.347040 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:41.349897 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.350695 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.350703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.350710 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.350851 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.351281 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.351287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.351291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.351667 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:41.351967 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.352363 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.352602 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:47:41.357581 15768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:47:41.364835 15768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:47:41.393749 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.378793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.5.71.130:34783)
0923 20:47:41.380239 (+ 1446us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.380249 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.380279 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:47:41.380383 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.380392 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.380685 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.383059 (+ 2374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:41.383345 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.388194 (+ 4849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.391153 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.392436 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.392605 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.392829 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.393185 (+ 356us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.393401 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":1170,"threads_started":1}
I20260923 20:47:41.393867 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.377942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:34783 (local address 127.0.0.1:59816)
0923 20:47:41.379461 (+ 1519us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:41.379498 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:41.379683 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:41.382217 (+ 2534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:41.382228 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:41.382270 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:41.382793 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.382810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.388359 (+ 5549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.388367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.390954 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.390971 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.392661 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.392667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.392935 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.392983 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 20:47:41.393130 (+ 147us) client_negotiation.cc:240] Negotiation successful
0923 20:47:41.393357 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"mutex_wait_us":33,"thread_start_us":1127,"threads_started":1}
I20260923 20:47:41.396646 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.378367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.5.71.131:34779)
0923 20:47:41.388174 (+ 9807us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.388182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.388216 (+ 34us) server_negotiation.cc:407] Connection header received
0923 20:47:41.388320 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.388330 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.388578 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.388803 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:41.390071 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.391618 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.394124 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.395494 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.395625 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.395880 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.396128 (+ 248us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.396298 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9634,"thread_start_us":9494,"threads_started":1}
I20260923 20:47:41.396670 16240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.378313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34779 (local address 127.0.0.1:45826)
0923 20:47:41.386341 (+ 8028us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:41.386368 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:41.387579 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:41.388798 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:41.388805 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:41.388839 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:41.389512 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.389524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.391780 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.391817 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.393949 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.393961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.395708 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.395714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.395911 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.395954 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 20:47:41.396059 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 20:47:41.396198 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7853,"mutex_wait_us":117,"thread_start_us":7686,"threads_started":1}
I20260923 20:47:41.397164 16242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.380463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:39853 (local address 127.0.0.1:60682)
0923 20:47:41.386771 (+ 6308us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:41.386796 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:41.386951 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:41.388422 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:41.388430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:41.388455 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:41.389014 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.389028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.391064 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.391073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.393373 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.393388 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.396012 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.396016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.396217 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.396252 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 20:47:41.396341 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 20:47:41.396481 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6137,"mutex_wait_us":33,"thread_start_us":5963,"threads_started":1}
I20260923 20:47:41.397889 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.381011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.5.71.129:39853)
0923 20:47:41.387689 (+ 6678us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.387702 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.387751 (+ 49us) server_negotiation.cc:407] Connection header received
0923 20:47:41.387922 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.387930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.388207 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.388457 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:41.389268 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.390933 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.393579 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.395494 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.395625 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.395880 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.397330 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.397491 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6449,"thread_start_us":6248,"threads_started":1}
I20260923 20:47:41.398440 16026 tablet_service.cc:1467] Processing CreateTablet for tablet 613f615f3c584cd7a1dedeb4e967ec27 (DEFAULT_TABLE table=test-table [id=24a5a0e9adee4e5da80b71645898c324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:41.399989 16026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613f615f3c584cd7a1dedeb4e967ec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:41.400504 16160 tablet_service.cc:1467] Processing CreateTablet for tablet 613f615f3c584cd7a1dedeb4e967ec27 (DEFAULT_TABLE table=test-table [id=24a5a0e9adee4e5da80b71645898c324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:41.401898 16160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613f615f3c584cd7a1dedeb4e967ec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:41.402733 15892 tablet_service.cc:1467] Processing CreateTablet for tablet 613f615f3c584cd7a1dedeb4e967ec27 (DEFAULT_TABLE table=test-table [id=24a5a0e9adee4e5da80b71645898c324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:47:41.403915 15892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613f615f3c584cd7a1dedeb4e967ec27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:47:41.415136 16244 tablet_bootstrap.cc:492] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Bootstrap starting.
I20260923 20:47:41.417213 16246 tablet_bootstrap.cc:492] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Bootstrap starting.
I20260923 20:47:41.420634 16246 tablet_bootstrap.cc:654] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:41.421545 16244 tablet_bootstrap.cc:654] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:41.421790 16246 log.cc:824] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:41.422564 16245 tablet_bootstrap.cc:492] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Bootstrap starting.
I20260923 20:47:41.423209 16244 log.cc:824] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:41.425462 16246 tablet_bootstrap.cc:492] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: No bootstrap required, opened a new log
I20260923 20:47:41.425763 16246 ts_tablet_manager.cc:1397] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260923 20:47:41.426553 16245 tablet_bootstrap.cc:654] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Neither blocks nor log segments found. Creating new log.
I20260923 20:47:41.427359 16244 tablet_bootstrap.cc:492] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: No bootstrap required, opened a new log
I20260923 20:47:41.427704 16244 ts_tablet_manager.cc:1397] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.006s
I20260923 20:47:41.427986 16245 log.cc:824] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Log is configured to *not* fsync() on all Append() calls
I20260923 20:47:41.431375 16245 tablet_bootstrap.cc:492] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: No bootstrap required, opened a new log
I20260923 20:47:41.431666 16245 ts_tablet_manager.cc:1397] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260923 20:47:41.433784 16246 raft_consensus.cc:348] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.434166 16246 raft_consensus.cc:374] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:41.434258 16246 raft_consensus.cc:729] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fb9e4c7cf042e6b4e12c08fa193c58, State: Initialized, Role: FOLLOWER
I20260923 20:47:41.434876 16246 consensus_queue.cc:260] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.437047 16225 heartbeater.cc:503] Master 127.5.71.190:34487 was elected leader, sending a full tablet report...
I20260923 20:47:41.437410 16246 ts_tablet_manager.cc:1428] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260923 20:47:41.442135 16244 raft_consensus.cc:348] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.442664 16244 raft_consensus.cc:374] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:41.442771 16244 raft_consensus.cc:729] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bf90afbc2b44c88163b2c3cb7f783f, State: Initialized, Role: FOLLOWER
I20260923 20:47:41.443336 16245 raft_consensus.cc:348] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.443478 16244 consensus_queue.cc:260] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.443830 16245 raft_consensus.cc:374] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:47:41.443946 16245 raft_consensus.cc:729] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1ddd2af45c4696ade550f9cdcdea11, State: Initialized, Role: FOLLOWER
I20260923 20:47:41.444721 16091 heartbeater.cc:503] Master 127.5.71.190:34487 was elected leader, sending a full tablet report...
I20260923 20:47:41.444626 16245 consensus_queue.cc:260] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.445662 16244 ts_tablet_manager.cc:1428] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Time spent starting tablet: real 0.018s user 0.013s sys 0.001s
I20260923 20:47:41.446622 16245 ts_tablet_manager.cc:1428] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260923 20:47:41.460103 15958 tablet.cc:2369] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:47:41.508610 16092 tablet.cc:2369] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:47:41.548549 16226 tablet.cc:2369] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:47:41.625792 16252 raft_consensus.cc:482] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:47:41.626140 16252 raft_consensus.cc:504] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.628201 16252 leader_election.cc:288] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04bf90afbc2b44c88163b2c3cb7f783f (127.5.71.130:34783), e4fb9e4c7cf042e6b4e12c08fa193c58 (127.5.71.131:34779)
I20260923 20:47:41.635619 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.628555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:41167 (local address 127.5.71.130:34783)
0923 20:47:41.628770 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.628780 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.629852 (+ 1072us) server_negotiation.cc:407] Connection header received
0923 20:47:41.629973 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.629981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.630061 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.630158 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:41.630829 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.631553 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.633694 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.634500 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.634711 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.634814 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.635203 (+ 389us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.635339 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:47:41.635656 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.628929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:48459 (local address 127.5.71.131:34779)
0923 20:47:41.629161 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.629167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.629597 (+ 430us) server_negotiation.cc:407] Connection header received
0923 20:47:41.629798 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.629804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.629892 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.630000 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:47:41.630636 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.631460 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.633614 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.634495 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.634716 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.634807 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.635252 (+ 445us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.635381 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260923 20:47:41.635869 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.628462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34779 (local address 127.5.71.129:48459)
0923 20:47:41.629453 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:41.629481 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:41.629641 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:41.630026 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:41.630031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:41.630056 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:41.630472 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.630482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.631600 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.631607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.633434 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.633449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.634814 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.634821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.635070 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.635093 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 20:47:41.635205 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 20:47:41.635342 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":70,"thread_start_us":620,"threads_started":1}
I20260923 20:47:41.635916 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.628594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:34783 (local address 127.5.71.129:41167)
0923 20:47:41.629671 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0923 20:47:41.629686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:47:41.629836 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:47:41.630210 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:47:41.630214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:47:41.630234 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:47:41.630654 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.630666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.631705 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.631709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.633487 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:47:41.633527 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.634760 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.634767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:47:41.635012 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.635062 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 20:47:41.635151 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 20:47:41.635319 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":1,"thread_start_us":745,"threads_started":1}
I20260923 20:47:41.637064 16180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613f615f3c584cd7a1dedeb4e967ec27" candidate_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" is_pre_election: true
I20260923 20:47:41.637100 16046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613f615f3c584cd7a1dedeb4e967ec27" candidate_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" is_pre_election: true
I20260923 20:47:41.637616 16046 raft_consensus.cc:2454] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1ddd2af45c4696ade550f9cdcdea11 in term 0.
I20260923 20:47:41.637624 16180 raft_consensus.cc:2454] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1ddd2af45c4696ade550f9cdcdea11 in term 0.
I20260923 20:47:41.638331 15845 leader_election.cc:302] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bf90afbc2b44c88163b2c3cb7f783f, bf1ddd2af45c4696ade550f9cdcdea11; no voters:
I20260923 20:47:41.638772 16252 raft_consensus.cc:2781] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:47:41.638944 16252 raft_consensus.cc:482] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:47:41.639051 16252 raft_consensus.cc:3037] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:41.641553 16252 raft_consensus.cc:504] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.642215 16252 leader_election.cc:288] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [CANDIDATE]: Term 1 election: Requested vote from peers 04bf90afbc2b44c88163b2c3cb7f783f (127.5.71.130:34783), e4fb9e4c7cf042e6b4e12c08fa193c58 (127.5.71.131:34779)
I20260923 20:47:41.642675 16046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613f615f3c584cd7a1dedeb4e967ec27" candidate_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bf90afbc2b44c88163b2c3cb7f783f"
I20260923 20:47:41.642796 16180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613f615f3c584cd7a1dedeb4e967ec27" candidate_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58"
I20260923 20:47:41.642913 16046 raft_consensus.cc:3037] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:41.642993 16180 raft_consensus.cc:3037] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:47:41.646237 16046 raft_consensus.cc:2454] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1ddd2af45c4696ade550f9cdcdea11 in term 1.
I20260923 20:47:41.646235 16180 raft_consensus.cc:2454] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1ddd2af45c4696ade550f9cdcdea11 in term 1.
I20260923 20:47:41.646710 15848 leader_election.cc:302] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04bf90afbc2b44c88163b2c3cb7f783f, bf1ddd2af45c4696ade550f9cdcdea11, e4fb9e4c7cf042e6b4e12c08fa193c58; no voters:
I20260923 20:47:41.647063 16252 raft_consensus.cc:2781] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:47:41.647573 16252 raft_consensus.cc:686] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 1 LEADER]: Becoming Leader. State: Replica: bf1ddd2af45c4696ade550f9cdcdea11, State: Running, Role: LEADER
I20260923 20:47:41.648208 16252 consensus_queue.cc:237] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } }
I20260923 20:47:41.653522 15768 catalog_manager.cc:5495] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf1ddd2af45c4696ade550f9cdcdea11 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf1ddd2af45c4696ade550f9cdcdea11" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 39853 } health_report { overall_health: HEALTHY } } }
I20260923 20:47:41.696637 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.690605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.5.71.129:39853)
0923 20:47:41.690888 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.690893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.690927 (+ 34us) server_negotiation.cc:407] Connection header received
0923 20:47:41.691159 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.691163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.691243 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.691412 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:47:41.691771 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.692602 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.694130 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.694571 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.694685 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.695216 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:47:41.695940 (+ 724us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:47:41.696128 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.696347 (+ 219us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.696422 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260923 20:47:41.705863 16180 raft_consensus.cc:1261] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 1 FOLLOWER]: Refusing update from remote peer bf1ddd2af45c4696ade550f9cdcdea11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:41.705862 16046 raft_consensus.cc:1261] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 1 FOLLOWER]: Refusing update from remote peer bf1ddd2af45c4696ade550f9cdcdea11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:47:41.706933 16255 consensus_queue.cc:1035] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4fb9e4c7cf042e6b4e12c08fa193c58" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:41.707270 16252 consensus_queue.cc:1035] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04bf90afbc2b44c88163b2c3cb7f783f" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:47:41.720594 16260 mvcc.cc:205] Tried to move back new op lower bound from 7332644707136163840 to 7332644706915213312. Current Snapshot: MvccSnapshot[applied={T|T < 7332644707136163840}]
I20260923 20:47:41.725421 16259 mvcc.cc:205] Tried to move back new op lower bound from 7332644707136163840 to 7332644706915213312. Current Snapshot: MvccSnapshot[applied={T|T < 7332644707136163840 or (T in {7332644707136163840})}]
I20260923 20:47:41.731884 15768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:41.732193 15768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:47:41.736873 15768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c77f50316dc54befa93fc58162597cb0: Sending DeleteTablet for 3 replicas of tablet 613f615f3c584cd7a1dedeb4e967ec27
I20260923 20:47:41.738266 16160 tablet_service.cc:1514] Processing DeleteTablet for tablet 613f615f3c584cd7a1dedeb4e967ec27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:41 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:45826
I20260923 20:47:41.738265 16026 tablet_service.cc:1514] Processing DeleteTablet for tablet 613f615f3c584cd7a1dedeb4e967ec27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:41 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:59816
I20260923 20:47:41.739046 15892 tablet_service.cc:1514] Processing DeleteTablet for tablet 613f615f3c584cd7a1dedeb4e967ec27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:47:41 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:60682
I20260923 20:47:41.739243 16267 tablet_replica.cc:330] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: stopping tablet replica
I20260923 20:47:41.739729 16267 raft_consensus.cc:2229] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:41.740101 16269 tablet_replica.cc:330] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: stopping tablet replica
I20260923 20:47:41.740334 16267 raft_consensus.cc:2258] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:41.741206 16268 tablet_replica.cc:330] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: stopping tablet replica
I20260923 20:47:41.741406 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15829
I20260923 20:47:41.741709 16268 raft_consensus.cc:2229] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:47:41.742098 16269 raft_consensus.cc:2229] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:47:41.742225 16268 raft_consensus.cc:2258] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:41.743453 16269 raft_consensus.cc:2258] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:47:41.745735 16269 ts_tablet_manager.cc:1905] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:41.746151 16267 ts_tablet_manager.cc:1905] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:41.754559 16268 ts_tablet_manager.cc:1905] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:47:41.757212 16269 ts_tablet_manager.cc:1918] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:41.757477 16269 log.cc:1196] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/613f615f3c584cd7a1dedeb4e967ec27
I20260923 20:47:41.757637 16267 ts_tablet_manager.cc:1918] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:41.757822 16267 log.cc:1196] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/613f615f3c584cd7a1dedeb4e967ec27
I20260923 20:47:41.758260 16269 ts_tablet_manager.cc:1939] T 613f615f3c584cd7a1dedeb4e967ec27 P bf1ddd2af45c4696ade550f9cdcdea11: Deleting consensus metadata
I20260923 20:47:41.758366 16267 ts_tablet_manager.cc:1939] T 613f615f3c584cd7a1dedeb4e967ec27 P e4fb9e4c7cf042e6b4e12c08fa193c58: Deleting consensus metadata
I20260923 20:47:41.762778 16268 ts_tablet_manager.cc:1918] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:47:41.763036 16268 log.cc:1196] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/613f615f3c584cd7a1dedeb4e967ec27
I20260923 20:47:41.763566 16268 ts_tablet_manager.cc:1939] T 613f615f3c584cd7a1dedeb4e967ec27 P 04bf90afbc2b44c88163b2c3cb7f783f: Deleting consensus metadata
I20260923 20:47:41.764068 15755 catalog_manager.cc:4841] TS e4fb9e4c7cf042e6b4e12c08fa193c58 (127.5.71.131:34779): tablet 613f615f3c584cd7a1dedeb4e967ec27 (table test-table [id=24a5a0e9adee4e5da80b71645898c324]) successfully deleted
I20260923 20:47:41.765143 15754 catalog_manager.cc:4841] TS bf1ddd2af45c4696ade550f9cdcdea11 (127.5.71.129:39853): tablet 613f615f3c584cd7a1dedeb4e967ec27 (table test-table [id=24a5a0e9adee4e5da80b71645898c324]) successfully deleted
I20260923 20:47:41.765318 15752 catalog_manager.cc:4841] TS 04bf90afbc2b44c88163b2c3cb7f783f (127.5.71.130:34783): tablet 613f615f3c584cd7a1dedeb4e967ec27 (table test-table [id=24a5a0e9adee4e5da80b71645898c324]) successfully deleted
Sep 23 20:47:41 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:47:41.779558 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.743193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.5.71.129:39853)
0923 20:47:41.743384 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.743391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.754290 (+ 10899us) server_negotiation.cc:407] Connection header received
0923 20:47:41.754734 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.754741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.754838 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.754927 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:41.756902 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.760778 (+ 3876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.762884 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.763585 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.763689 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.771167 (+ 7478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:41.771184 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:41.771205 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:41.771264 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:41.775834 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.776810 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.776820 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.776824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.777010 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.777517 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.777522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.777526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.778004 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:41.778336 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.778657 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.778910 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 20:47:41.780390 15952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:41.881507 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15829
I20260923 20:47:41.900755 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15963
Sep 23 20:47:41 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:47:41.916836 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:41.901597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.5.71.130:34783)
0923 20:47:41.901763 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:41.901770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:41.901793 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:47:41.901933 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:41.901938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:41.902001 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:41.902083 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:41.903363 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.904215 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.905515 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:41.905945 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:41.906052 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:41.910357 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:41.910371 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:41.910389 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:41.910432 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:41.913828 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.914607 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.914615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.914617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.914821 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:41.915237 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:41.915240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:41.915243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:41.915649 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:41.916056 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:41.916324 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 20:47:41.916496 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 20:47:41.917575 16086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:42.026288 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15963
I20260923 20:47:42.043556 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 16097
Sep 23 20:47:42 dist-test-slave-dn29 krb5kdc[15723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:47:42.060482 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:47:42.044474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.5.71.131:34779)
0923 20:47:42.044660 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 20:47:42.044667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:47:42.044688 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:47:42.044856 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:47:42.044863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:47:42.044951 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:47:42.045060 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:47:42.046367 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:42.047056 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:42.048476 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:47:42.048870 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:47:42.049006 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:47:42.053704 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:47:42.053721 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:47:42.053740 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:47:42.053782 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:47:42.056983 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:42.057848 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:42.057856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:42.057859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:42.058038 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:47:42.058507 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:47:42.058513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:47:42.058516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:47:42.058903 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:47:42.059166 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0923 20:47:42.059614 (+ 448us) server_negotiation.cc:299] Negotiation successful
0923 20:47:42.060125 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 20:47:42.061474 16220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:42.152774 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 16097
I20260923 20:47:42.169907 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 15736
I20260923 20:47:42.171116 15798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:47:42.281407 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 15736
2026-09-23T20:47:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4621 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:47:42 dist-test-slave-dn29 krb5kdc[16277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:47:42 dist-test-slave-dn29 krb5kdc[16277](info): set up 2 sockets
Sep 23 20:47:42 dist-test-slave-dn29 krb5kdc[16277](info): commencing operation
krb5kdc: starting...
W20260923 20:47:43.405704 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:47:43 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:47:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/rangeradmin_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/rangeradmin_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/rangerlookup_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/rangerlookup_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
I20260923 20:47:43.764845 5406 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQIJ804/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/postgres -l logfile start
2026-09-23 20:47:46.797 UTC [16307] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 20:47:46.797 UTC [16307] LOG: listening on IPv4 address "127.5.71.148", port 43305
2026-09-23 20:47:46.801 UTC [16307] LOG: listening on Unix socket "/tmp/.s.PGSQL.43305"
2026-09-23 20:47:46.827 UTC [16310] LOG: database system was shut down at 2026-09-23 20:47:45 UTC
2026-09-23 20:47:46.832 UTC [16307] LOG: database system is ready to accept connections
I20260923 20:47:47.840893 5406 mini_postgres.cc:96] Postgres bound to 43305
2026-09-23 20:47:47.860 UTC [16318] FATAL: database "slave" does not exist
127.5.71.148:43305 - accepting connections
I20260923 20:47:47.861259 5406 mini_ranger.cc:161] Starting Ranger...
I20260923 20:47:47.891050 5406 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 20:47:48.533459 5406 mini_ranger.cc:87] Created ranger Postgres database
I20260923 20:47:48.533599 5406 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-admin
2026-09-23 20:47:49,441 [I] DB FLAVOR :POSTGRES
2026-09-23 20:47:49,441 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 20:47:49,441 [I] Checking connection..
2026-09-23 20:47:49,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 20:47:49,905 [I] Checking connection passed.
2026-09-23 20:47:49,906 [I] --------- Verifying version history table ---------
2026-09-23 20:47:49,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 20:47:50,197 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 20:47:50,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 20:47:50,489 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 20:47:50,489 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 20:47:50,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 20:47:50.776 UTC [16382] WARNING: there is no transaction in progress
2026-09-23 20:47:50,784 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 20:47:50,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 20:47:51,038 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 20:47:51,038 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 20:47:51,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 20:47:51,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 20:47:51,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-dn29.c.gcp-upstream.internal','N') ;"
2026-09-23 20:47:51,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 20:47:52,081 [I] Table x_portal_user does not exist in database ranger
2026-09-23 20:47:52,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 20:47:52,361 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 20:47:52,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 20:47:52,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 20:47:52,887 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 20:47:52,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 20:47:54.026 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.047 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.066 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.085 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.105 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.127 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.199 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.208 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.226 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.239 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.251 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.266 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.279 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.287 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.296 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:54.305 UTC [16490] WARNING: there is no transaction in progress
2026-09-23 20:47:55,122 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 20:47:55,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 20:47:55,378 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 20:47:55,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 20:47:55,639 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 20:47:55,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 20:47:55,894 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 20:47:55,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 20:47:56,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 20:47:56,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-dn29.c.gcp-upstream.internal';"
2026-09-23 20:47:56,680 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 20:47:56,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 20:47:56,920 [I] DB_PATCHES have already been applied
I20260923 20:47:56.927152 5406 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQIJ804/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 20:47:56.927253 5406 mini_ranger.cc:193] Using host: 127.5.71.148
I20260923 20:47:56.943264 5406 mini_ranger.cc:227] Ranger admin URL: http://127.5.71.148:52693
Sep 23, 2026 8:47:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 8:47:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 8:47:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 8:47:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 8:47:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 8:47:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 8:47:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 20:47:59.268050 5406 mini_ranger.cc:160] Time spent starting Ranger: real 11.407s user 0.001s sys 0.053s
Sep 23, 2026 8:48:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 8:48:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 8:48:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 8:48:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 8:48:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 20:48:23.764009 5406 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/rangerkms_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/rangerkms_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.71.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.71.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 20:48:23 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196503, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 20:48:23.957849 5406 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 20:48:23.982079 5406 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 20:48:25.097193 5406 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 20:48:25.097419 5406 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms
I20260923 20:48:25.097553 5406 mini_ranger_kms.cc:227] Using postgres at 127.5.71.148:43305
2026-09-23 20:48:26,010 [I] DB FLAVOR :POSTGRES
2026-09-23 20:48:26,010 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 20:48:26,011 [I] Checking connection
2026-09-23 20:48:26,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 20:48:26,285 [I] connection success
2026-09-23 20:48:26,286 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 20:48:26,286 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 20:48:26,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 20:48:26,559 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 20:48:26,559 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 20:48:26,560 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 20:48:26,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:43305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 20:48:26,825 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 20:48:27.342505 5406 mini_ranger_kms.cc:310] Created kms service
I20260923 20:48:27.560925 5406 mini_ranger_kms.cc:326] Created kudu user
I20260923 20:48:27.612599 5406 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 20:48:27 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196503, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:48:28.042541 5406 mini_ranger_kms.cc:384] Added ranger policy
I20260923 20:48:28.042900 5406 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 20:48:28.042977 5406 mini_ranger_kms.cc:240] Using host: 127.5.71.148
I20260923 20:48:28.061528 5406 mini_ranger_kms.cc:276] Ranger KMS PID: 16672
I20260923 20:48:28.061769 5406 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.71.148:59075
Sep 23, 2026 8:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 8:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 8:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 8:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 8:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 8:48:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 20:48:30.390974 5406 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.433s user 0.141s sys 0.417s
I20260923 20:48:30.391263 5406 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 20:48:30.391499 5406 mini_ranger_kms.cc:401] 127.5.71.148:59075/kms/v1/keys
Sep 23 20:48:30 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790196510, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.71.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 20:48:30 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196510, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.71.148@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
Sep 23 20:48:31 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196510, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.71.148@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
Sep 23 20:48:33 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:48:33.929382 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45383
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45383
--ranger_config_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:59075/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:48:34.364140 16722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:48:34.364570 16722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:48:34.364760 16722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:48:34.373441 16722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:48:34.373644 16722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:48:34.373757 16722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:48:34.373852 16722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:48:34.373960 16722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:48:34.386646 16722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.71.148:59075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45383
--ranger_config_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:48:34.388825 16722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:48:34.390363 16722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:48:34.405999 16728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:34.406139 16727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:34.406086 16730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:48:34.406464 16722 server_base.cc:1034] running on GCE node
I20260923 20:48:34.408336 16722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:48:34.410423 16722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:48:34.411972 16722 hybrid_clock.cc:648] HybridClock initialized: now 1790196514411898 us; error 101 us; skew 500 ppm
I20260923 20:48:34.412500 16722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:48:34 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196514, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:48:34.416468 16722 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:48:34.419049 16722 webserver.cc:460] Webserver started at http://127.5.71.190:41533/ using document root <none> and password file <none>
I20260923 20:48:34.419938 16722 fs_manager.cc:359] Metadata directory not provided
I20260923 20:48:34.420106 16722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:48:34.420469 16722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:48:34 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196514, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:48:34.509932 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "8ef028615d984a1d82711d9ebe5717ba"
format_stamp: "Formatted at 2026-09-23 20:48:34 on dist-test-slave-dn29"
server_key: "2e2d794893325d9789239f94aca55188"
server_key_iv: "b5ca84d2e753420aa006f7967205478b"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:34.511731 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "8ef028615d984a1d82711d9ebe5717ba"
format_stamp: "Formatted at 2026-09-23 20:48:34 on dist-test-slave-dn29"
server_key: "2e2d794893325d9789239f94aca55188"
server_key_iv: "b5ca84d2e753420aa006f7967205478b"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:34.524086 16722 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.010s sys 0.000s
I20260923 20:48:34.552805 16737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:34.556180 16722 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.005s sys 0.000s
I20260923 20:48:34.556506 16722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "8ef028615d984a1d82711d9ebe5717ba"
format_stamp: "Formatted at 2026-09-23 20:48:34 on dist-test-slave-dn29"
server_key: "2e2d794893325d9789239f94aca55188"
server_key_iv: "b5ca84d2e753420aa006f7967205478b"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:34.556839 16722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:48:34.586490 16722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:48:34.594087 16722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:48:34.594431 16722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:48:34.629065 16722 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45383
I20260923 20:48:34.629259 16788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45383 every 8 connection(s)
I20260923 20:48:34.631031 16722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:48:34.632485 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 16722
I20260923 20:48:34.632740 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:48:34.634071 5406 external_mini_cluster.cc:1428] Setting key 04075362b91877bda309b5be868f7ba2
I20260923 20:48:34.640293 16789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:48:34 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:48:34.660888 16789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba: Bootstrap starting.
I20260923 20:48:34.664952 16789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba: Neither blocks nor log segments found. Creating new log.
I20260923 20:48:34.666393 16789 log.cc:824] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba: Log is configured to *not* fsync() on all Append() calls
I20260923 20:48:34.668490 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:34.637009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.5.71.190:45383)
0923 20:48:34.638944 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:34.638961 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:34.639045 (+ 84us) server_negotiation.cc:407] Connection header received
0923 20:48:34.643431 (+ 4386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:34.643480 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:34.644197 (+ 717us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:34.644837 (+ 640us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:34.646309 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:34.649775 (+ 3466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:34.651353 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:34.653015 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:34.653282 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:34.657909 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:34.657925 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:34.657961 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:34.658028 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:34.662531 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:34.663524 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:34.663541 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:34.663549 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:34.663860 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:34.664340 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:34.664347 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:34.664351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:34.664961 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:34.665270 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:34.665967 (+ 697us) server_negotiation.cc:299] Negotiation successful
0923 20:48:34.666508 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1401,"thread_start_us":976,"threads_started":1}
I20260923 20:48:34.670560 16789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba: No bootstrap required, opened a new log
I20260923 20:48:34.678812 16789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef028615d984a1d82711d9ebe5717ba" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45383 } }
I20260923 20:48:34.679352 16789 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:48:34.679514 16789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef028615d984a1d82711d9ebe5717ba, State: Initialized, Role: FOLLOWER
I20260923 20:48:34.680315 16789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef028615d984a1d82711d9ebe5717ba" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45383 } }
I20260923 20:48:34.680519 16789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:48:34.680672 16789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:48:34.680860 16789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:48:34.683130 16789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef028615d984a1d82711d9ebe5717ba" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45383 } }
I20260923 20:48:34.683676 16789 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef028615d984a1d82711d9ebe5717ba; no voters:
I20260923 20:48:34.684355 16789 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:48:34.684508 16794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:48:34.686264 16794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [term 1 LEADER]: Becoming Leader. State: Replica: 8ef028615d984a1d82711d9ebe5717ba, State: Running, Role: LEADER
I20260923 20:48:34.686825 16794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef028615d984a1d82711d9ebe5717ba" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45383 } }
I20260923 20:48:34.686982 16789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:48:34.690135 16796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef028615d984a1d82711d9ebe5717ba. Latest consensus state: current_term: 1 leader_uuid: "8ef028615d984a1d82711d9ebe5717ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef028615d984a1d82711d9ebe5717ba" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45383 } } }
I20260923 20:48:34.690174 16795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ef028615d984a1d82711d9ebe5717ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef028615d984a1d82711d9ebe5717ba" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45383 } } }
I20260923 20:48:34.690544 16796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [sys.catalog]: This master's current role is: LEADER
I20260923 20:48:34.690596 16795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba [sys.catalog]: This master's current role is: LEADER
I20260923 20:48:34.694371 16803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:48:34.701005 16803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:48:34.701779 16789 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 20:48:34.721866 16803 catalog_manager.cc:1347] Generated new cluster ID: be9cec6240a14dd3bbbc66d58d533cf9
I20260923 20:48:34.722038 16803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:48:34.734820 16803 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:48:34.737049 16803 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:48:34.757360 16803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba: Generated new TSK 0
I20260923 20:48:34.758716 16803 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:48:35.617882 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45383
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:48:35 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790196515, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 20:48:36.125878 16834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:48:36.126298 16834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:48:36.126515 16834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:48:36.137290 16834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:48:36.137564 16834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:48:36.137862 16834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:48:36.154628 16834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.71.148:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:48:36.156319 16834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:48:36.157970 16834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:48:36.178741 16844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:36.179423 16845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:48:36.186863 16834 server_base.cc:1034] running on GCE node
W20260923 20:48:36.187393 16847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:48:36.188434 16834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:48:36.192096 16834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:48:36.193495 16834 hybrid_clock.cc:648] HybridClock initialized: now 1790196516193421 us; error 72 us; skew 500 ppm
I20260923 20:48:36.194007 16834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:48:36 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196516, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:48:36.197491 16834 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:48:36.200210 16834 webserver.cc:460] Webserver started at http://127.5.71.129:34679/ using document root <none> and password file <none>
I20260923 20:48:36.201006 16834 fs_manager.cc:359] Metadata directory not provided
I20260923 20:48:36.201160 16834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:48:36.201516 16834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:48:36 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196516, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:48:36.230881 16834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "68453202c4dd42238d8c2fa910140bb2"
format_stamp: "Formatted at 2026-09-23 20:48:36 on dist-test-slave-dn29"
server_key: "f7eaa138202b7248b313ff1c11c17ee1"
server_key_iv: "f4c72f364c3bd8031a8a8ca4d4c86118"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:36.232302 16834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "68453202c4dd42238d8c2fa910140bb2"
format_stamp: "Formatted at 2026-09-23 20:48:36 on dist-test-slave-dn29"
server_key: "f7eaa138202b7248b313ff1c11c17ee1"
server_key_iv: "f4c72f364c3bd8031a8a8ca4d4c86118"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:36.249615 16834 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.014s sys 0.001s
I20260923 20:48:36.284744 16854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:36.287315 16834 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.004s
I20260923 20:48:36.287736 16834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "68453202c4dd42238d8c2fa910140bb2"
format_stamp: "Formatted at 2026-09-23 20:48:36 on dist-test-slave-dn29"
server_key: "f7eaa138202b7248b313ff1c11c17ee1"
server_key_iv: "f4c72f364c3bd8031a8a8ca4d4c86118"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:36.288177 16834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:48:36.312429 16834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:48:36.318295 16834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:48:36.318691 16834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:48:36.320199 16834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:48:36.322548 16834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:48:36.322670 16834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:36.322899 16834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:48:36.322990 16834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:48:36 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196515, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:48:36.591220 16834 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:45179
I20260923 20:48:36.591634 16966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:45179 every 8 connection(s)
I20260923 20:48:36.593273 16834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:48:36.602253 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 16834
I20260923 20:48:36.602571 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:48:36.603994 5406 external_mini_cluster.cc:1428] Setting key ddc08b120a0158629939d5363beb54cb
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Sep 23 20:48:36 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196516, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:48:36.687606 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:36.599345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:56591 (local address 127.5.71.190:45383)
0923 20:48:36.647387 (+ 48042us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:36.647398 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:36.647429 (+ 31us) server_negotiation.cc:407] Connection header received
0923 20:48:36.647510 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:36.647520 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:36.647674 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:36.664094 (+ 16420us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:36.664437 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:36.666511 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:36.670311 (+ 3800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:36.670841 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:36.671019 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:36.676264 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:36.676272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:36.676277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:36.676326 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:36.681231 (+ 4905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:36.684247 (+ 3016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:36.684255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:36.684260 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:36.684455 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:36.685111 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:36.685117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:36.685122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:36.685519 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:36.685680 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:36.686879 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0923 20:48:36.687162 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47813,"thread_start_us":57474,"threads_started":1}
I20260923 20:48:36.690347 16970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:36.600137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45383 (local address 127.5.71.129:56591)
0923 20:48:36.605136 (+ 4999us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:36.605197 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:36.609623 (+ 4426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:36.648558 (+ 38935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:36.648575 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:36.652309 (+ 3734us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:48:36.653814 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:36.653853 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:36.667026 (+ 13173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:36.667037 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:36.669993 (+ 2956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:36.670022 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:36.671194 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:36.671207 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:36.671565 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:36.672420 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:48:36.672464 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:48:36.675955 (+ 3491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:48:36.683203 (+ 7248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:48:36.683217 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:48:36.683243 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:48:36.684036 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:48:36.684649 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:48:36.684670 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:48:36.684674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:48:36.684960 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:48:36.685800 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:48:36.685810 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:48:36.686330 (+ 520us) client_negotiation.cc:769] Sending connection context
0923 20:48:36.688264 (+ 1934us) client_negotiation.cc:240] Negotiation successful
0923 20:48:36.688903 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4500,"thread_start_us":4036,"threads_started":1}
I20260923 20:48:36.692973 16967 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45383
I20260923 20:48:36.693372 16967 heartbeater.cc:464] Registering TS with master...
I20260923 20:48:36.694357 16967 heartbeater.cc:511] Master 127.5.71.190:45383 requested a full tablet report, sending...
I20260923 20:48:36.697295 16754 ts_manager.cc:194] Registered new tserver with Master: 68453202c4dd42238d8c2fa910140bb2 (127.5.71.129:45179)
I20260923 20:48:36.699375 16754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:56591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
W20260923 20:48:36.719605 5406 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.71.130: real 0.111s user 0.002s sys 0.019s
I20260923 20:48:36.720752 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45383
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:48:36 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196515, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
W20260923 20:48:37.223663 16973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:48:37.224238 16973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:48:37.224470 16973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:48:37.237177 16973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:48:37.237399 16973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:48:37.237668 16973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:48:37.249404 16973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.71.148:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:48:37.250900 16973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:48:37.252357 16973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:48:37.270426 16981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:37.271119 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:37.271368 16983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:48:37.273270 16973 server_base.cc:1034] running on GCE node
I20260923 20:48:37.273998 16973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:48:37.284541 16973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:48:37.288075 16973 hybrid_clock.cc:648] HybridClock initialized: now 1790196517287925 us; error 143 us; skew 500 ppm
I20260923 20:48:37.288604 16973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:48:37 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196517, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:48:37.293768 16973 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:48:37.303387 16973 webserver.cc:460] Webserver started at http://127.5.71.130:40301/ using document root <none> and password file <none>
I20260923 20:48:37.304374 16973 fs_manager.cc:359] Metadata directory not provided
I20260923 20:48:37.304500 16973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:48:37.304919 16973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:48:37 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196517, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:48:37.325063 16973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "b5b97337c144459e88da166c2dca8093"
format_stamp: "Formatted at 2026-09-23 20:48:37 on dist-test-slave-dn29"
server_key: "0e3a1b3633664365dc3e63abfb832250"
server_key_iv: "6be6c39e07ff2919413eabdd6f068431"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:37.326324 16973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "b5b97337c144459e88da166c2dca8093"
format_stamp: "Formatted at 2026-09-23 20:48:37 on dist-test-slave-dn29"
server_key: "0e3a1b3633664365dc3e63abfb832250"
server_key_iv: "6be6c39e07ff2919413eabdd6f068431"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:37.334208 16973 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260923 20:48:37.354377 16997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:37.356616 16973 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 20:48:37.356849 16973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "b5b97337c144459e88da166c2dca8093"
format_stamp: "Formatted at 2026-09-23 20:48:37 on dist-test-slave-dn29"
server_key: "0e3a1b3633664365dc3e63abfb832250"
server_key_iv: "6be6c39e07ff2919413eabdd6f068431"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:37.357134 16973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:48:37.392676 16973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:48:37.399639 16973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:48:37.400041 16973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:48:37.401384 16973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:48:37.403424 16973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:48:37.403529 16973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:37.403688 16973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:48:37.403761 16973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:37.515303 16973 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:41169
I20260923 20:48:37.515420 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:41169 every 8 connection(s)
I20260923 20:48:37.517787 16973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:48:37.521407 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 16973
I20260923 20:48:37.521715 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:48:37.522749 5406 external_mini_cluster.cc:1428] Setting key 2410311c194c694ff6144981d1a9087a
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Sep 23 20:48:37 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196517, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:48:37.586784 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:37.539337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45383 (local address 127.5.71.130:55005)
0923 20:48:37.553832 (+ 14495us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:37.553900 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:37.556363 (+ 2463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:37.564298 (+ 7935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:37.564314 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:37.565036 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:48:37.565907 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:37.565935 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:37.567499 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:37.567505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:37.569738 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:37.569755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:37.570937 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:37.570945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:37.571257 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:37.572187 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:48:37.572221 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:48:37.576608 (+ 4387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:48:37.581807 (+ 5199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:48:37.581819 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:48:37.581843 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:48:37.582497 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:48:37.583002 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:48:37.583007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:48:37.583011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:48:37.583298 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:48:37.584051 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:48:37.584061 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:48:37.584510 (+ 449us) client_negotiation.cc:769] Sending connection context
0923 20:48:37.584814 (+ 304us) client_negotiation.cc:240] Negotiation successful
0923 20:48:37.585369 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12048,"thread_start_us":12354,"threads_started":1}
I20260923 20:48:37.587985 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:37.540180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:55005 (local address 127.5.71.190:45383)
0923 20:48:37.563474 (+ 23294us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:37.563481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:37.563508 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:48:37.563601 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:37.563607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:37.563699 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:37.563873 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:37.566169 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:37.567298 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:37.569928 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:37.570643 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:37.570776 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:37.576956 (+ 6180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:37.576963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:37.576968 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:37.577012 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:37.581587 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:37.582673 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:37.582680 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:37.582684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:37.582829 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:37.583435 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:37.583440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:37.583443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:37.583815 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:37.583978 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:37.587309 (+ 3331us) server_negotiation.cc:299] Negotiation successful
0923 20:48:37.587558 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23149,"thread_start_us":22881,"threads_started":1}
I20260923 20:48:37.590235 17110 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45383
I20260923 20:48:37.590553 17110 heartbeater.cc:464] Registering TS with master...
I20260923 20:48:37.591503 17110 heartbeater.cc:511] Master 127.5.71.190:45383 requested a full tablet report, sending...
I20260923 20:48:37.593628 16754 ts_manager.cc:194] Registered new tserver with Master: b5b97337c144459e88da166c2dca8093 (127.5.71.130:41169)
I20260923 20:48:37.594808 16754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:55005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:48:37.612699 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45383
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:48:37.703658 16967 heartbeater.cc:503] Master 127.5.71.190:45383 was elected leader, sending a full tablet report...
W20260923 20:48:38.126164 17116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:48:38.126705 17116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:48:38.126986 17116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:48:38.139189 17116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:48:38.139544 17116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:48:38.139926 17116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:48:38.154702 17116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.71.148:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:48:38.156553 17116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:48:38.158281 17116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:48:38.175406 17121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:38.177590 17124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:48:38.177677 17122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:48:38.179738 17116 server_base.cc:1034] running on GCE node
I20260923 20:48:38.180576 17116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:48:38.181666 17116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:48:38.182924 17116 hybrid_clock.cc:648] HybridClock initialized: now 1790196518182859 us; error 65 us; skew 500 ppm
I20260923 20:48:38.183282 17116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:48:38 dist-test-slave-dn29 krb5kdc[16277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196518, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:48:38.188663 17116 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:48:38.192034 17116 webserver.cc:460] Webserver started at http://127.5.71.131:37949/ using document root <none> and password file <none>
I20260923 20:48:38.193075 17116 fs_manager.cc:359] Metadata directory not provided
I20260923 20:48:38.193338 17116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:48:38.193800 17116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:48:38 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196518, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:48:38.214077 17116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "96276f607e6947b7aac1b54f72446281"
format_stamp: "Formatted at 2026-09-23 20:48:38 on dist-test-slave-dn29"
server_key: "dae95220bea244d46abc59605ac9ef30"
server_key_iv: "c8dcdea7e7d585c5d9ab22747a7b13ec"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:38.215601 17116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "96276f607e6947b7aac1b54f72446281"
format_stamp: "Formatted at 2026-09-23 20:48:38 on dist-test-slave-dn29"
server_key: "dae95220bea244d46abc59605ac9ef30"
server_key_iv: "c8dcdea7e7d585c5d9ab22747a7b13ec"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:38.225675 17116 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.006s
I20260923 20:48:38.245729 17131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:38.247670 17116 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260923 20:48:38.248008 17116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "96276f607e6947b7aac1b54f72446281"
format_stamp: "Formatted at 2026-09-23 20:48:38 on dist-test-slave-dn29"
server_key: "dae95220bea244d46abc59605ac9ef30"
server_key_iv: "c8dcdea7e7d585c5d9ab22747a7b13ec"
server_key_version: "kuduclusterkey@0"
I20260923 20:48:38.248293 17116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:48:38.266161 17116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:48:38.270457 17116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:48:38.270716 17116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:48:38.272101 17116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:48:38.273972 17116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:48:38.274065 17116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:38.274155 17116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:48:38.274200 17116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:48:38.360435 17116 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:36513
I20260923 20:48:38.360620 17243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:36513 every 8 connection(s)
I20260923 20:48:38.361958 17116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:48:38.369788 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 17116
I20260923 20:48:38.369993 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:48:38.370447 5406 external_mini_cluster.cc:1428] Setting key f0c3780a94886efe4096734a70e3c51a
Sep 23 20:48:38 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196518, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:48:38.386904 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.366245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:42813 (local address 127.5.71.190:45383)
0923 20:48:38.367235 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.367240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.369442 (+ 2202us) server_negotiation.cc:407] Connection header received
0923 20:48:38.372283 (+ 2841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.372289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.372399 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.372532 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:38.374367 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.375077 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.376811 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.377242 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.377381 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.380850 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:38.380855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:38.380858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:38.380901 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:38.383423 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:38.384199 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:38.384203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:38.384206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:38.384327 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:38.384881 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:38.384885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:38.384887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:38.385295 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:38.385449 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.386351 (+ 902us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.386585 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":624,"threads_started":1}
I20260923 20:48:38.388077 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.367272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45383 (local address 127.5.71.131:42813)
0923 20:48:38.369257 (+ 1985us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:38.369304 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:38.371967 (+ 2663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:38.372803 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:38.372812 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:38.373394 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:48:38.374200 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.374218 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.375247 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.375251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.376627 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.376639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.377481 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.377488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.377710 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.378393 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:48:38.378420 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:48:38.380619 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:48:38.383557 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:48:38.383565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:48:38.383579 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:48:38.384078 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:48:38.384471 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:48:38.384475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:48:38.384479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:48:38.384762 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:48:38.385428 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:48:38.385436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:48:38.385873 (+ 437us) client_negotiation.cc:769] Sending connection context
0923 20:48:38.386192 (+ 319us) client_negotiation.cc:240] Negotiation successful
0923 20:48:38.386710 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1635,"thread_start_us":1192,"threads_started":1}
I20260923 20:48:38.389369 17244 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45383
I20260923 20:48:38.389690 17244 heartbeater.cc:464] Registering TS with master...
I20260923 20:48:38.390437 17244 heartbeater.cc:511] Master 127.5.71.190:45383 requested a full tablet report, sending...
I20260923 20:48:38.392010 16754 ts_manager.cc:194] Registered new tserver with Master: 96276f607e6947b7aac1b54f72446281 (127.5.71.131:36513)
I20260923 20:48:38.392868 16754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:42813
I20260923 20:48:38.399899 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:48:38.418634 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.404940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.5.71.190:45383)
0923 20:48:38.405128 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.405132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.405757 (+ 625us) server_negotiation.cc:407] Connection header received
0923 20:48:38.405893 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.405897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.406000 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.406128 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:38.407569 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.408377 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.409922 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.410486 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.410624 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.413160 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:38.413165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:38.413169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:38.413202 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:38.415892 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:38.416764 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:38.416769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:38.416771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:38.416910 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:38.417353 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:38.417357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:38.417370 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:38.417689 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:38.417901 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.418190 (+ 289us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.418385 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:48:38.423661 16754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:48:38.434346 16754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:48:38.473839 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.454030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.5.71.129:45179)
0923 20:48:38.455621 (+ 1591us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.455630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.458432 (+ 2802us) server_negotiation.cc:407] Connection header received
0923 20:48:38.458864 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.458872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.459335 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.459582 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:48:38.461716 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.464243 (+ 2527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.469596 (+ 5353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.471517 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.471691 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.472261 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.472872 (+ 611us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.473155 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1291,"thread_start_us":1557,"threads_started":1}
I20260923 20:48:38.473901 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.454258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45179 (local address 127.0.0.1:60038)
0923 20:48:38.458280 (+ 4022us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:38.458327 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:38.458701 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:38.460190 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:38.460199 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:38.460227 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:48:38.461192 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.461223 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.464498 (+ 3275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.464507 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.469395 (+ 4888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.469407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.472240 (+ 2833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.472251 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.472515 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.472558 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 20:48:38.472805 (+ 247us) client_negotiation.cc:240] Negotiation successful
0923 20:48:38.473077 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3631,"spinlock_wait_cycles":3968,"thread_start_us":3640,"threads_started":1}
I20260923 20:48:38.478999 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.458580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:41169 (local address 127.0.0.1:36850)
0923 20:48:38.461908 (+ 3328us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:38.461929 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:38.462124 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:38.463059 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:38.463064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:38.463099 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:48:38.466144 (+ 3045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.466153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.468233 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.468241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.472939 (+ 4698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.472971 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.476058 (+ 3087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.476065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.477975 (+ 1910us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.478011 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 20:48:38.478113 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 20:48:38.478260 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3121,"mutex_wait_us":26,"thread_start_us":2854,"threads_started":1}
I20260923 20:48:38.479473 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.454363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:36513 (local address 127.0.0.1:48604)
0923 20:48:38.458300 (+ 3937us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:38.458327 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:38.458553 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:38.463772 (+ 5219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:38.463777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:38.463876 (+ 99us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:48:38.466128 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.466144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.469294 (+ 3150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.469300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.475300 (+ 6000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.475318 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.477954 (+ 2636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.477962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.478297 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.478343 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 20:48:38.478466 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 20:48:38.478611 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3793,"mutex_wait_us":25,"spinlock_wait_cycles":768,"thread_start_us":3054,"threads_started":1}
I20260923 20:48:38.480253 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.455147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.5.71.131:36513)
0923 20:48:38.457057 (+ 1910us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.457073 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.458501 (+ 1428us) server_negotiation.cc:407] Connection header received
0923 20:48:38.462449 (+ 3948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.462458 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.462796 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.463058 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:48:38.466438 (+ 3380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.469007 (+ 2569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.475547 (+ 6540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.477594 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.477808 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.478243 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.478534 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.478847 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1547,"thread_start_us":1194,"threads_started":1}
I20260923 20:48:38.480278 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.458964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.5.71.130:41169)
0923 20:48:38.462418 (+ 3454us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.462427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.462454 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:48:38.462553 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.462558 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.462834 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.463058 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:48:38.466468 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.468088 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.473248 (+ 5160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.475622 (+ 2374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.475892 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.476278 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.479319 (+ 3041us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.479606 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3271,"thread_start_us":3080,"threads_started":1}
I20260923 20:48:38.479650 16902 tablet_service.cc:1467] Processing CreateTablet for tablet aba6332b35ea4892ae72c3f558f55f43 (DEFAULT_TABLE table=test-table [id=20b8df6289924ee4beb5ffe02f9f2874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:48:38.482028 16902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba6332b35ea4892ae72c3f558f55f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:48:38.484772 17045 tablet_service.cc:1467] Processing CreateTablet for tablet aba6332b35ea4892ae72c3f558f55f43 (DEFAULT_TABLE table=test-table [id=20b8df6289924ee4beb5ffe02f9f2874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:48:38.486656 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba6332b35ea4892ae72c3f558f55f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:48:38.485497 17179 tablet_service.cc:1467] Processing CreateTablet for tablet aba6332b35ea4892ae72c3f558f55f43 (DEFAULT_TABLE table=test-table [id=20b8df6289924ee4beb5ffe02f9f2874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:48:38.488533 17179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba6332b35ea4892ae72c3f558f55f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:48:38.506014 17263 tablet_bootstrap.cc:492] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Bootstrap starting.
I20260923 20:48:38.507467 17264 tablet_bootstrap.cc:492] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Bootstrap starting.
I20260923 20:48:38.517321 17264 tablet_bootstrap.cc:654] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Neither blocks nor log segments found. Creating new log.
I20260923 20:48:38.521823 17264 log.cc:824] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Log is configured to *not* fsync() on all Append() calls
I20260923 20:48:38.522069 17265 tablet_bootstrap.cc:492] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Bootstrap starting.
I20260923 20:48:38.527422 17265 tablet_bootstrap.cc:654] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Neither blocks nor log segments found. Creating new log.
I20260923 20:48:38.529237 17263 tablet_bootstrap.cc:654] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Neither blocks nor log segments found. Creating new log.
I20260923 20:48:38.530434 17265 log.cc:824] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Log is configured to *not* fsync() on all Append() calls
I20260923 20:48:38.532282 17263 log.cc:824] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Log is configured to *not* fsync() on all Append() calls
I20260923 20:48:38.533142 17264 tablet_bootstrap.cc:492] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: No bootstrap required, opened a new log
I20260923 20:48:38.533681 17264 ts_tablet_manager.cc:1397] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.011s
I20260923 20:48:38.539669 17265 tablet_bootstrap.cc:492] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: No bootstrap required, opened a new log
I20260923 20:48:38.540248 17265 ts_tablet_manager.cc:1397] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.015s
I20260923 20:48:38.548933 17263 tablet_bootstrap.cc:492] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: No bootstrap required, opened a new log
I20260923 20:48:38.549484 17263 ts_tablet_manager.cc:1397] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.018s
I20260923 20:48:38.551259 17264 raft_consensus.cc:348] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.552197 17264 raft_consensus.cc:374] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:48:38.552399 17264 raft_consensus.cc:729] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68453202c4dd42238d8c2fa910140bb2, State: Initialized, Role: FOLLOWER
I20260923 20:48:38.552660 17265 raft_consensus.cc:348] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.553323 17265 raft_consensus.cc:374] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:48:38.553498 17265 raft_consensus.cc:729] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96276f607e6947b7aac1b54f72446281, State: Initialized, Role: FOLLOWER
I20260923 20:48:38.553576 17264 consensus_queue.cc:260] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.554309 17265 consensus_queue.cc:260] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.556919 17264 ts_tablet_manager.cc:1428] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Time spent starting tablet: real 0.023s user 0.013s sys 0.009s
I20260923 20:48:38.558576 17244 heartbeater.cc:503] Master 127.5.71.190:45383 was elected leader, sending a full tablet report...
I20260923 20:48:38.559964 17265 ts_tablet_manager.cc:1428] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Time spent starting tablet: real 0.019s user 0.016s sys 0.001s
I20260923 20:48:38.565196 17263 raft_consensus.cc:348] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.565652 17263 raft_consensus.cc:374] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:48:38.565770 17263 raft_consensus.cc:729] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b97337c144459e88da166c2dca8093, State: Initialized, Role: FOLLOWER
I20260923 20:48:38.566366 17263 consensus_queue.cc:260] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.567963 17110 heartbeater.cc:503] Master 127.5.71.190:45383 was elected leader, sending a full tablet report...
I20260923 20:48:38.568368 17263 ts_tablet_manager.cc:1428] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Time spent starting tablet: real 0.019s user 0.012s sys 0.008s
W20260923 20:48:38.615422 17245 tablet.cc:2369] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:48:38.654174 16968 tablet.cc:2369] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:48:38.788415 17111 tablet.cc:2369] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:48:38.846561 17271 raft_consensus.cc:482] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:48:38.847028 17271 raft_consensus.cc:504] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.849103 17271 leader_election.cc:288] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68453202c4dd42238d8c2fa910140bb2 (127.5.71.129:45179), 96276f607e6947b7aac1b54f72446281 (127.5.71.131:36513)
I20260923 20:48:38.856501 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.849640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:42573 (local address 127.5.71.131:36513)
0923 20:48:38.849796 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.849803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.850850 (+ 1047us) server_negotiation.cc:407] Connection header received
0923 20:48:38.851100 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.851104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.851163 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.851270 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:48:38.852046 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.852855 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.854713 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.855520 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.855713 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.855792 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.856103 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.856243 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 20:48:38.857048 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.849440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:36513 (local address 127.5.71.130:42573)
0923 20:48:38.850724 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:38.850739 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:38.850952 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:38.851280 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:38.851285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:38.851321 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:48:38.851893 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.851902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.852995 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.853001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.854574 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.854585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.855738 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.855745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.855942 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.855967 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 20:48:38.856077 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 20:48:38.856216 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"mutex_wait_us":21,"spinlock_wait_cycles":29568,"thread_start_us":844,"threads_started":1}
I20260923 20:48:38.857420 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.849554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:38385 (local address 127.5.71.129:45179)
0923 20:48:38.849811 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.849820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.850830 (+ 1010us) server_negotiation.cc:407] Connection header received
0923 20:48:38.851061 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.851069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.851135 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.851251 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:48:38.852035 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.853354 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.855365 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.856292 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.856495 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.856578 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.856975 (+ 397us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.857129 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260923 20:48:38.857522 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.849331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45179 (local address 127.5.71.130:38385)
0923 20:48:38.850659 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0923 20:48:38.850678 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:48:38.850900 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:48:38.851267 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:48:38.851271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:48:38.851295 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:48:38.851882 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.851893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.853497 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.853504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.855197 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:48:38.855209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.856528 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.856535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:48:38.856717 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.856742 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 20:48:38.856859 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 20:48:38.857015 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1199,"mutex_wait_us":29,"thread_start_us":917,"threads_started":1}
I20260923 20:48:38.858309 17199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba6332b35ea4892ae72c3f558f55f43" candidate_uuid: "b5b97337c144459e88da166c2dca8093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96276f607e6947b7aac1b54f72446281" is_pre_election: true
I20260923 20:48:38.858594 16922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba6332b35ea4892ae72c3f558f55f43" candidate_uuid: "b5b97337c144459e88da166c2dca8093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68453202c4dd42238d8c2fa910140bb2" is_pre_election: true
I20260923 20:48:38.858799 17199 raft_consensus.cc:2454] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b97337c144459e88da166c2dca8093 in term 0.
I20260923 20:48:38.859115 16922 raft_consensus.cc:2454] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b97337c144459e88da166c2dca8093 in term 0.
I20260923 20:48:38.859510 16998 leader_election.cc:302] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96276f607e6947b7aac1b54f72446281, b5b97337c144459e88da166c2dca8093; no voters:
I20260923 20:48:38.860109 17271 raft_consensus.cc:2781] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:48:38.860280 17271 raft_consensus.cc:482] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:48:38.860410 17271 raft_consensus.cc:3037] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:48:38.863269 17271 raft_consensus.cc:504] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.864156 17271 leader_election.cc:288] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [CANDIDATE]: Term 1 election: Requested vote from peers 68453202c4dd42238d8c2fa910140bb2 (127.5.71.129:45179), 96276f607e6947b7aac1b54f72446281 (127.5.71.131:36513)
I20260923 20:48:38.864727 16922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba6332b35ea4892ae72c3f558f55f43" candidate_uuid: "b5b97337c144459e88da166c2dca8093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68453202c4dd42238d8c2fa910140bb2"
I20260923 20:48:38.864897 17199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba6332b35ea4892ae72c3f558f55f43" candidate_uuid: "b5b97337c144459e88da166c2dca8093" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96276f607e6947b7aac1b54f72446281"
I20260923 20:48:38.864976 16922 raft_consensus.cc:3037] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:48:38.865128 17199 raft_consensus.cc:3037] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:48:38.868410 17199 raft_consensus.cc:2454] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b97337c144459e88da166c2dca8093 in term 1.
I20260923 20:48:38.868412 16922 raft_consensus.cc:2454] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b97337c144459e88da166c2dca8093 in term 1.
I20260923 20:48:38.868917 16998 leader_election.cc:302] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96276f607e6947b7aac1b54f72446281, b5b97337c144459e88da166c2dca8093; no voters:
I20260923 20:48:38.869305 17271 raft_consensus.cc:2781] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:48:38.869839 17271 raft_consensus.cc:686] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 1 LEADER]: Becoming Leader. State: Replica: b5b97337c144459e88da166c2dca8093, State: Running, Role: LEADER
I20260923 20:48:38.870369 17271 consensus_queue.cc:237] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } }
I20260923 20:48:38.875651 16753 catalog_manager.cc:5495] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5b97337c144459e88da166c2dca8093 (127.5.71.130). New cstate: current_term: 1 leader_uuid: "b5b97337c144459e88da166c2dca8093" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5b97337c144459e88da166c2dca8093" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 41169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 } health_report { overall_health: UNKNOWN } } }
I20260923 20:48:38.927603 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.920555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.5.71.130:41169)
0923 20:48:38.920834 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.920844 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.921287 (+ 443us) server_negotiation.cc:407] Connection header received
0923 20:48:38.921591 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.921595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.921688 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.921874 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:48:38.922355 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.923195 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.924960 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.925478 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:38.925640 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:38.926230 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:48:38.926887 (+ 657us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:48:38.927067 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:38.927157 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 20:48:38.927263 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260923 20:48:38.936815 17199 raft_consensus.cc:1261] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 1 FOLLOWER]: Refusing update from remote peer b5b97337c144459e88da166c2dca8093: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:48:38.936977 16922 raft_consensus.cc:1261] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 1 FOLLOWER]: Refusing update from remote peer b5b97337c144459e88da166c2dca8093: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:48:38.938014 17274 consensus_queue.cc:1035] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96276f607e6947b7aac1b54f72446281" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 36513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:48:38.938472 17271 consensus_queue.cc:1035] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68453202c4dd42238d8c2fa910140bb2" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:48:38.953214 17278 mvcc.cc:205] Tried to move back new op lower bound from 7332644941554466816 to 7332644941297336320. Current Snapshot: MvccSnapshot[applied={T|T < 7332644941554466816}]
I20260923 20:48:38.971957 16754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:48:38.972298 16754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:48:38.976378 16754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ef028615d984a1d82711d9ebe5717ba: Sending DeleteTablet for 3 replicas of tablet aba6332b35ea4892ae72c3f558f55f43
I20260923 20:48:38.978042 16902 tablet_service.cc:1514] Processing DeleteTablet for tablet aba6332b35ea4892ae72c3f558f55f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:48:38 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:60038
I20260923 20:48:38.978055 17179 tablet_service.cc:1514] Processing DeleteTablet for tablet aba6332b35ea4892ae72c3f558f55f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:48:38 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:48604
I20260923 20:48:38.978835 17045 tablet_service.cc:1514] Processing DeleteTablet for tablet aba6332b35ea4892ae72c3f558f55f43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:48:38 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260923 20:48:38.979604 17288 tablet_replica.cc:330] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: stopping tablet replica
I20260923 20:48:38.979918 17290 tablet_replica.cc:330] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: stopping tablet replica
I20260923 20:48:38.980237 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 16834
I20260923 20:48:38.980337 17288 raft_consensus.cc:2229] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:48:38.980381 17290 raft_consensus.cc:2229] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:48:38.980880 17288 raft_consensus.cc:2258] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:48:38.981104 17290 raft_consensus.cc:2258] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:48:38.981642 17289 tablet_replica.cc:330] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: stopping tablet replica
I20260923 20:48:38.982061 17289 raft_consensus.cc:2229] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:48:38.982518 17289 raft_consensus.cc:2258] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:48:38.983399 17290 ts_tablet_manager.cc:1905] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:48:38.984613 17289 ts_tablet_manager.cc:1905] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:48:38.991778 17288 ts_tablet_manager.cc:1905] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:48:38.992470 17289 ts_tablet_manager.cc:1918] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:48:38.992837 17289 log.cc:1196] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/aba6332b35ea4892ae72c3f558f55f43
I20260923 20:48:38.993628 17289 ts_tablet_manager.cc:1939] T aba6332b35ea4892ae72c3f558f55f43 P 96276f607e6947b7aac1b54f72446281: Deleting consensus metadata
I20260923 20:48:38.994736 17290 ts_tablet_manager.cc:1918] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:48:38.994913 17290 log.cc:1196] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/aba6332b35ea4892ae72c3f558f55f43
I20260923 20:48:38.996095 16738 catalog_manager.cc:4841] TS 96276f607e6947b7aac1b54f72446281 (127.5.71.131:36513): tablet aba6332b35ea4892ae72c3f558f55f43 (table test-table [id=20b8df6289924ee4beb5ffe02f9f2874]) successfully deleted
I20260923 20:48:38.998980 17290 ts_tablet_manager.cc:1939] T aba6332b35ea4892ae72c3f558f55f43 P b5b97337c144459e88da166c2dca8093: Deleting consensus metadata
I20260923 20:48:39.000013 17288 ts_tablet_manager.cc:1918] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:48:39.000314 17288 log.cc:1196] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/aba6332b35ea4892ae72c3f558f55f43
I20260923 20:48:39.000967 17288 ts_tablet_manager.cc:1939] T aba6332b35ea4892ae72c3f558f55f43 P 68453202c4dd42238d8c2fa910140bb2: Deleting consensus metadata
I20260923 20:48:39.001384 16739 catalog_manager.cc:4841] TS b5b97337c144459e88da166c2dca8093 (127.5.71.130:41169): tablet aba6332b35ea4892ae72c3f558f55f43 (table test-table [id=20b8df6289924ee4beb5ffe02f9f2874]) successfully deleted
I20260923 20:48:39.002938 16739 catalog_manager.cc:4841] TS 68453202c4dd42238d8c2fa910140bb2 (127.5.71.129:45179): tablet aba6332b35ea4892ae72c3f558f55f43 (table test-table [id=20b8df6289924ee4beb5ffe02f9f2874]) successfully deleted
Sep 23 20:48:39 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:48:39.014389 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:38.983698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.5.71.129:45179)
0923 20:48:38.983895 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:38.983903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:38.994093 (+ 10190us) server_negotiation.cc:407] Connection header received
0923 20:48:38.994680 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:38.994686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:38.994783 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:38.994871 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:38.998337 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:38.999605 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:39.001800 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:39.002281 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:39.002390 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:39.007482 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:39.007500 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:39.007516 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:39.007557 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:39.010701 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:39.011704 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:39.011712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:39.011715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:39.011929 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:39.012431 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:39.012435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:39.012438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:39.012964 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:39.013230 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:39.013640 (+ 410us) server_negotiation.cc:299] Negotiation successful
0923 20:48:39.013906 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 20:48:39.015198 16962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:48:39.111013 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 16834
I20260923 20:48:39.129519 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 16973
Sep 23 20:48:39 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:48:39.147310 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:39.130554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.5.71.130:41169)
0923 20:48:39.130761 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:39.130770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:39.130792 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:48:39.130858 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:39.130864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:39.130942 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:39.131075 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:39.132572 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:39.133452 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:39.134811 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:39.135321 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:39.135470 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:39.140230 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:39.140247 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:39.140268 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:39.140317 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:39.144137 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:39.144978 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:39.144989 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:39.144992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:39.145161 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:39.145575 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:39.145580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:39.145584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:39.146012 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:39.146352 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:39.146674 (+ 322us) server_negotiation.cc:299] Negotiation successful
0923 20:48:39.146934 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 20:48:39.148049 17105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:48:39.265697 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 16973
I20260923 20:48:39.281852 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 17116
Sep 23 20:48:39 dist-test-slave-dn29 krb5kdc[16277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:48:39.299482 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:48:39.282728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.5.71.131:36513)
0923 20:48:39.283045 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0923 20:48:39.283051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:48:39.283073 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:48:39.283296 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:48:39.283300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:48:39.283369 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:48:39.283483 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:48:39.285054 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:39.285761 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:39.287057 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:48:39.287591 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:48:39.287744 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:48:39.293176 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:48:39.293193 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:48:39.293213 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:48:39.293270 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:48:39.296469 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:39.297263 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:39.297271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:39.297274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:39.297413 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:48:39.297825 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:48:39.297835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:48:39.297837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:48:39.298222 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:48:39.298538 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0923 20:48:39.298887 (+ 349us) server_negotiation.cc:299] Negotiation successful
0923 20:48:39.299112 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260923 20:48:39.300275 17239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:48:39.391891 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 17116
I20260923 20:48:39.409178 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 16722
I20260923 20:48:39.410593 16784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:48:39.525439 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 16722
I20260923 20:48:39.551331 5406 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 20:48:39.898201 5406 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 20:48:39.898451 5406 mini_ranger.cc:66] Stopping Ranger...
I20260923 20:48:40.246161 5406 mini_ranger.cc:68] Stopped Ranger
2026-09-23 20:48:40.246 UTC [16307] LOG: received smart shutdown request
2026-09-23 20:48:40.251 UTC [16307] LOG: background worker "logical replication launcher" (PID 16316) exited with exit code 1
2026-09-23 20:48:40.251 UTC [16311] LOG: shutting down
2026-09-23 20:48:40.455 UTC [16307] LOG: database system is shut down
2026-09-23T20:48:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (58306 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:48:40 dist-test-slave-dn29 krb5kdc[17304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:48:40 dist-test-slave-dn29 krb5kdc[17304](info): set up 2 sockets
Sep 23 20:48:40 dist-test-slave-dn29 krb5kdc[17304](info): commencing operation
krb5kdc: starting...
W20260923 20:48:41.725201 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:48:41 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:48:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/rangeradmin_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/rangeradmin_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/rangerlookup_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/rangerlookup_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
I20260923 20:48:42.085052 5406 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQIJ804/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/postgres -l logfile start
2026-09-23 20:48:45.168 UTC [17334] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 20:48:45.168 UTC [17334] LOG: listening on IPv4 address "127.5.71.148", port 57755
2026-09-23 20:48:45.173 UTC [17334] LOG: listening on Unix socket "/tmp/.s.PGSQL.57755"
2026-09-23 20:48:45.197 UTC [17337] LOG: database system was shut down at 2026-09-23 20:48:43 UTC
2026-09-23 20:48:45.202 UTC [17334] LOG: database system is ready to accept connections
I20260923 20:48:46.208166 5406 mini_postgres.cc:96] Postgres bound to 57755
2026-09-23 20:48:46.227 UTC [17345] FATAL: database "slave" does not exist
127.5.71.148:57755 - accepting connections
I20260923 20:48:46.228305 5406 mini_ranger.cc:161] Starting Ranger...
I20260923 20:48:46.257059 5406 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 20:48:46.916666 5406 mini_ranger.cc:87] Created ranger Postgres database
I20260923 20:48:46.916910 5406 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-admin
2026-09-23 20:48:47,246 [I] DB FLAVOR :POSTGRES
2026-09-23 20:48:47,246 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 20:48:47,246 [I] Checking connection..
2026-09-23 20:48:47,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 20:48:47,507 [I] Checking connection passed.
2026-09-23 20:48:47,507 [I] --------- Verifying version history table ---------
2026-09-23 20:48:47,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 20:48:47,756 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 20:48:47,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 20:48:48,011 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 20:48:48,011 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 20:48:48,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 20:48:48.294 UTC [17409] WARNING: there is no transaction in progress
2026-09-23 20:48:48,302 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 20:48:48,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 20:48:48,569 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 20:48:48,569 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 20:48:48,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 20:48:48,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 20:48:49,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-dn29.c.gcp-upstream.internal','N') ;"
2026-09-23 20:48:49,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 20:48:49,671 [I] Table x_portal_user does not exist in database ranger
2026-09-23 20:48:49,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 20:48:49,945 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 20:48:49,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 20:48:50,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 20:48:50,484 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 20:48:50,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 20:48:51.660 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.681 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.702 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.722 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.741 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.765 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.837 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.848 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.869 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.884 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.897 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.914 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.927 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.936 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.945 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:51.955 UTC [17517] WARNING: there is no transaction in progress
2026-09-23 20:48:52,893 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 20:48:52,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 20:48:53,193 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 20:48:53,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 20:48:53,473 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 20:48:53,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 20:48:53,738 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 20:48:53,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 20:48:54,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 20:48:54,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-dn29.c.gcp-upstream.internal';"
2026-09-23 20:48:54,559 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 20:48:54,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 20:48:54,815 [I] DB_PATCHES have already been applied
I20260923 20:48:54.822093 5406 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQIJ804/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 20:48:54.822283 5406 mini_ranger.cc:193] Using host: 127.5.71.148
I20260923 20:48:54.838249 5406 mini_ranger.cc:227] Ranger admin URL: http://127.5.71.148:40607
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 8:48:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 20:48:56.042721 5406 mini_ranger.cc:160] Time spent starting Ranger: real 9.814s user 0.000s sys 0.043s
Sep 23, 2026 8:49:19 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 8:49:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 8:49:19 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 8:49:19 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 8:49:20 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 20:49:21.187372 5406 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.71.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.71.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/rangerkms_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.71.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/rangerkms_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.71.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.71.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.71.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/HTTP_127.5.71.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 20:49:21 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196561, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 20:49:21.392921 5406 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 20:49:21.418103 5406 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 20:49:22.612982 5406 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 20:49:22.613098 5406 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms
I20260923 20:49:22.613152 5406 mini_ranger_kms.cc:227] Using postgres at 127.5.71.148:57755
2026-09-23 20:49:23,000 [I] DB FLAVOR :POSTGRES
2026-09-23 20:49:23,001 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 20:49:23,001 [I] Checking connection
2026-09-23 20:49:23,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 20:49:23,270 [I] connection success
2026-09-23 20:49:23,270 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 20:49:23,270 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 20:49:23,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 20:49:23,544 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 20:49:23,544 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 20:49:23,544 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 20:49:23,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.71.148:57755/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 20:49:23,812 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 20:49:24.366117 5406 mini_ranger_kms.cc:310] Created kms service
I20260923 20:49:24.544377 5406 mini_ranger_kms.cc:326] Created kudu user
I20260923 20:49:24.591243 5406 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 20:49:24 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196561, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:25.039125 5406 mini_ranger_kms.cc:384] Added ranger policy
I20260923 20:49:25.039484 5406 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQIJ804/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQIJ804/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 20:49:25.039556 5406 mini_ranger_kms.cc:240] Using host: 127.5.71.148
I20260923 20:49:25.058678 5406 mini_ranger_kms.cc:276] Ranger KMS PID: 17698
I20260923 20:49:25.058837 5406 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.71.148:56553
Sep 23, 2026 8:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 8:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 8:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 8:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 8:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 8:49:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 20:49:27.366369 5406 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.973s user 0.071s sys 0.325s
I20260923 20:49:27.366644 5406 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 20:49:27.366832 5406 mini_ranger_kms.cc:401] 127.5.71.148:56553/kms/v1/keys
Sep 23 20:49:27 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790196567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.71.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 20:49:27 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.71.148@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
Sep 23 20:49:28 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196567, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.71.148@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
Sep 23 20:49:30 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:49:30.796917 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42921
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:42921
--ranger_config_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:31.153586 17748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:31.153898 17748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:31.154047 17748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:31.162274 17748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:31.162395 17748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:31.162477 17748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:31.162577 17748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:31.162652 17748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:31.173452 17748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.71.148:56553/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:42921
--ranger_config_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:31.174865 17748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:31.176347 17748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:31.191885 17753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:31.191766 17754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:31.192171 17756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:31.193097 17748 server_base.cc:1034] running on GCE node
I20260923 20:49:31.194206 17748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:31.196270 17748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:31.197552 17748 hybrid_clock.cc:648] HybridClock initialized: now 1790196571197489 us; error 55 us; skew 500 ppm
I20260923 20:49:31.197988 17748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:31 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:31.207508 17748 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:31.209426 17748 webserver.cc:460] Webserver started at http://127.5.71.190:33949/ using document root <none> and password file <none>
I20260923 20:49:31.210204 17748 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:31.210383 17748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:31.210745 17748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:49:31 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196571, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:31.261758 17748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "8bcd15417eac444faedbce35875821e2"
format_stamp: "Formatted at 2026-09-23 20:49:31 on dist-test-slave-dn29"
server_key: "1c2822b8180e8baee16da7e299c42777"
server_key_iv: "625d78984ee7aabc8a33fc1f0ef53644"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:31.262823 17748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "8bcd15417eac444faedbce35875821e2"
format_stamp: "Formatted at 2026-09-23 20:49:31 on dist-test-slave-dn29"
server_key: "1c2822b8180e8baee16da7e299c42777"
server_key_iv: "625d78984ee7aabc8a33fc1f0ef53644"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:31.268923 17748 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:49:31.289656 17763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:31.291383 17748 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:49:31.291582 17748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "8bcd15417eac444faedbce35875821e2"
format_stamp: "Formatted at 2026-09-23 20:49:31 on dist-test-slave-dn29"
server_key: "1c2822b8180e8baee16da7e299c42777"
server_key_iv: "625d78984ee7aabc8a33fc1f0ef53644"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:31.291867 17748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:31.314154 17748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:31.318663 17748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:31.319008 17748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:31.339401 17748 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:42921
I20260923 20:49:31.339424 17814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:42921 every 8 connection(s)
I20260923 20:49:31.340942 17748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:31.344659 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 17748
I20260923 20:49:31.344877 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:49:31.345204 17815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:31.345690 5406 external_mini_cluster.cc:1428] Setting key 360208923224a184cb478dc8b3ee0d5d
I20260923 20:49:31.364657 17815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Bootstrap starting.
I20260923 20:49:31.368656 17815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:31.369953 17815 log.cc:824] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:31.373301 17815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: No bootstrap required, opened a new log
Sep 23 20:49:31 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:31.382537 17815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:31.383884 17815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:31.384024 17815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bcd15417eac444faedbce35875821e2, State: Initialized, Role: FOLLOWER
I20260923 20:49:31.384819 17815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:31.385128 17815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:49:31.385337 17815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:49:31.385545 17815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:31.387920 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:31.347888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.5.71.190:42921)
0923 20:49:31.362807 (+ 14919us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:31.362826 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:31.362885 (+ 59us) server_negotiation.cc:407] Connection header received
0923 20:49:31.365314 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:31.365345 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:31.365897 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:31.366348 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:31.368078 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:31.370106 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:31.372002 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:31.373043 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:31.373215 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:31.378365 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:31.378379 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:31.378403 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:31.378462 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:31.382506 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:31.383491 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:31.383501 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:31.383509 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:31.383705 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:31.384213 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:31.384218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:31.384221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:31.384962 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:31.385289 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:31.385913 (+ 624us) server_negotiation.cc:299] Negotiation successful
0923 20:49:31.386392 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14646,"thread_start_us":14429,"threads_started":1}
I20260923 20:49:31.389008 17815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:31.389670 17815 leader_election.cc:302] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bcd15417eac444faedbce35875821e2; no voters:
I20260923 20:49:31.390385 17815 leader_election.cc:288] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:49:31.390518 17820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:31.394289 17815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:31.394640 17820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 LEADER]: Becoming Leader. State: Replica: 8bcd15417eac444faedbce35875821e2, State: Running, Role: LEADER
I20260923 20:49:31.395391 17820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:31.398442 17822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bcd15417eac444faedbce35875821e2. Latest consensus state: current_term: 1 leader_uuid: "8bcd15417eac444faedbce35875821e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } } }
I20260923 20:49:31.398816 17822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:31.399362 17821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bcd15417eac444faedbce35875821e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } } }
I20260923 20:49:31.399622 17821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:31.409888 17826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:31.415585 17826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:31.435401 17826 catalog_manager.cc:1347] Generated new cluster ID: 03c4093272274f2298c8979978f4e605
I20260923 20:49:31.435712 17826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:49:31.446298 17815 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 20:49:31.469101 17826 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:49:31.471266 17826 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:49:31.482921 17826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Generated new TSK 0
I20260923 20:49:31.484355 17826 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:49:32.397328 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42921
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:49:32 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790196572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 20:49:32.877113 17860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:32.877621 17860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:32.877830 17860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:32.889886 17860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:32.890038 17860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:32.890292 17860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:49:32.906677 17860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.71.148:56553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:32.908461 17860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:32.910264 17860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:32.942118 17871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:32.943593 17870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:32.946954 17873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:32.947752 17860 server_base.cc:1034] running on GCE node
I20260923 20:49:32.948608 17860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:32.949906 17860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:32.951256 17860 hybrid_clock.cc:648] HybridClock initialized: now 1790196572951185 us; error 57 us; skew 500 ppm
I20260923 20:49:32.951731 17860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:32 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:32.956367 17860 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:49:32.961328 17860 webserver.cc:460] Webserver started at http://127.5.71.129:40701/ using document root <none> and password file <none>
I20260923 20:49:32.962368 17860 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:32.962522 17860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:32.962934 17860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:49:32 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:32.982389 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "d05f60f7aba143489f8df618141ae0eb"
format_stamp: "Formatted at 2026-09-23 20:49:32 on dist-test-slave-dn29"
server_key: "2abb5f3082967eb783163b74072cd92a"
server_key_iv: "2fb0e51cad145e8778879b32c0fa6bfa"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:32.983593 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "d05f60f7aba143489f8df618141ae0eb"
format_stamp: "Formatted at 2026-09-23 20:49:32 on dist-test-slave-dn29"
server_key: "2abb5f3082967eb783163b74072cd92a"
server_key_iv: "2fb0e51cad145e8778879b32c0fa6bfa"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:32.992275 17860 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260923 20:49:33.028672 17880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:33.032063 17860 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.006s sys 0.003s
I20260923 20:49:33.032356 17860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "d05f60f7aba143489f8df618141ae0eb"
format_stamp: "Formatted at 2026-09-23 20:49:32 on dist-test-slave-dn29"
server_key: "2abb5f3082967eb783163b74072cd92a"
server_key_iv: "2fb0e51cad145e8778879b32c0fa6bfa"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:33.032675 17860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:33.084689 17860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:33.090608 17860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:33.091044 17860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:33.093539 17860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:33.095765 17860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:33.095949 17860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260923 20:49:33.096170 17860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:33.096287 17860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:33.172091 17860 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:44237
I20260923 20:49:33.172344 17992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:44237 every 8 connection(s)
I20260923 20:49:33.174726 17860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:49:33.184574 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 17860
I20260923 20:49:33.184872 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:49:33.185985 5406 external_mini_cluster.cc:1428] Setting key 0091751aa8bc549da93c115e2d06f300
Sep 23 20:49:33 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
Sep 23 20:49:33 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:33.229689 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:33.192002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:37491 (local address 127.5.71.190:42921)
0923 20:49:33.195109 (+ 3107us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:33.195117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:33.195141 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:49:33.198863 (+ 3722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:33.198870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:33.198990 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:33.199111 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:33.204297 (+ 5186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.205650 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.208514 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.209270 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.209385 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:33.214606 (+ 5221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:33.214614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:33.214619 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:33.214679 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:33.218694 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:33.220155 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:33.220162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:33.220167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:33.220334 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:33.221503 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:33.221509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:33.221513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:33.224903 (+ 3390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:33.225092 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:33.228798 (+ 3706us) server_negotiation.cc:299] Negotiation successful
0923 20:49:33.229081 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2975,"thread_start_us":2833,"threads_started":1}
I20260923 20:49:33.232296 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:33.191248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42921 (local address 127.5.71.129:37491)
0923 20:49:33.192891 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:33.192955 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:33.198290 (+ 5335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:33.201356 (+ 3066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:33.201372 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:33.202112 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:33.203972 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:33.204013 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.205915 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.205921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:33.208305 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:33.208323 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.209807 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.209815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:33.210153 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:33.211009 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:33.211052 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:33.214271 (+ 3219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:33.218949 (+ 4678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:33.218961 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:33.218983 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:33.219679 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:33.220508 (+ 829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:33.220514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:33.220519 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:33.220803 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:33.225219 (+ 4416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:33.225230 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:33.225724 (+ 494us) client_negotiation.cc:769] Sending connection context
0923 20:49:33.229991 (+ 4267us) client_negotiation.cc:240] Negotiation successful
0923 20:49:33.230682 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"thread_start_us":991,"threads_started":1}
I20260923 20:49:33.234567 17993 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42921
I20260923 20:49:33.234956 17993 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:33.235965 17993 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
I20260923 20:49:33.238807 17780 ts_manager.cc:194] Registered new tserver with Master: d05f60f7aba143489f8df618141ae0eb (127.5.71.129:44237)
I20260923 20:49:33.241104 17780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:37491
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:49:33.287855 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42921
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:49:33 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196572, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
W20260923 20:49:33.712958 17999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:33.713486 17999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:33.713716 17999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:33.725414 17999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:33.725801 17999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:33.726063 17999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:49:33.741959 17999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.71.148:56553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:33.743983 17999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:33.745790 17999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:33.776996 18007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:33.782943 18006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:33.784298 18010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:33.784593 17999 server_base.cc:1034] running on GCE node
I20260923 20:49:33.785439 17999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:33.786752 17999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:33.788043 17999 hybrid_clock.cc:648] HybridClock initialized: now 1790196573787923 us; error 103 us; skew 500 ppm
I20260923 20:49:33.788439 17999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:33 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:33.792838 17999 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:49:33.794893 17999 webserver.cc:460] Webserver started at http://127.5.71.130:44581/ using document root <none> and password file <none>
I20260923 20:49:33.795926 17999 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:33.796069 17999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:33.796442 17999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:49:33 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:33.817581 17999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "6c60a7580673462f86fda9114ceafac6"
format_stamp: "Formatted at 2026-09-23 20:49:33 on dist-test-slave-dn29"
server_key: "97c5e0cd61db2ba75ee1506fb3af1dbb"
server_key_iv: "a08a565de3291ae74f07bd1f36acaedd"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:33.818954 17999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "6c60a7580673462f86fda9114ceafac6"
format_stamp: "Formatted at 2026-09-23 20:49:33 on dist-test-slave-dn29"
server_key: "97c5e0cd61db2ba75ee1506fb3af1dbb"
server_key_iv: "a08a565de3291ae74f07bd1f36acaedd"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:33.826849 17999 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260923 20:49:33.868547 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:33.870734 17999 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.004s
I20260923 20:49:33.870924 17999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "6c60a7580673462f86fda9114ceafac6"
format_stamp: "Formatted at 2026-09-23 20:49:33 on dist-test-slave-dn29"
server_key: "97c5e0cd61db2ba75ee1506fb3af1dbb"
server_key_iv: "a08a565de3291ae74f07bd1f36acaedd"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:33.871186 17999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:33.906189 17999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:33.911727 17999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:33.912194 17999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:33.913522 17999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:33.915367 17999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:33.915467 17999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:33.915633 17999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:33.915700 17999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:33.957132 17999 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38139
I20260923 20:49:33.957213 18135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38139 every 8 connection(s)
I20260923 20:49:33.958791 17999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:49:33.964344 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 17999
I20260923 20:49:33.964529 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:49:33.965121 5406 external_mini_cluster.cc:1428] Setting key bdefcae74bf1018d74cb7a4599853791
Sep 23 20:49:33 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196573, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:34.002203 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:33.961835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:40531 (local address 127.5.71.190:42921)
0923 20:49:33.977639 (+ 15804us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:33.977646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:33.977671 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:49:33.979856 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:33.979864 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:33.979962 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:33.980076 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:33.982427 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.983775 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.986915 (+ 3140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.987699 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.987868 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:33.993261 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:33.993272 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:33.993280 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:33.993338 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:33.997401 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:33.998621 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:33.998631 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:33.998637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:33.998816 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:33.999462 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:33.999470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:33.999476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:33.999967 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:34.000117 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:34.001393 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0923 20:49:34.001676 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15614,"thread_start_us":15496,"threads_started":1}
I20260923 20:49:34.005278 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:33.962435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42921 (local address 127.5.71.130:40531)
0923 20:49:33.976715 (+ 14280us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:33.976796 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:33.979334 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:33.980551 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:33.980565 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:33.981266 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:33.982171 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:33.982203 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.984328 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.984336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:33.986677 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:33.986713 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:33.988287 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:33.988297 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:33.988655 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:33.989572 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:33.989611 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:33.992890 (+ 3279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:33.997642 (+ 4752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:33.997657 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:33.997685 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:33.998445 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:33.998992 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:33.999000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:33.999006 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:33.999317 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:34.000230 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:34.000243 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:34.000804 (+ 561us) client_negotiation.cc:769] Sending connection context
0923 20:49:34.002876 (+ 2072us) client_negotiation.cc:240] Negotiation successful
0923 20:49:34.003568 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13833,"thread_start_us":13503,"threads_started":1}
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:49:34.007352 18136 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42921
I20260923 20:49:34.007751 18136 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:34.008512 18136 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
I20260923 20:49:34.010342 17780 ts_manager.cc:194] Registered new tserver with Master: 6c60a7580673462f86fda9114ceafac6 (127.5.71.130:38139)
I20260923 20:49:34.011206 17780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:40531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:49:34.040234 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42921
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:34.245296 17993 heartbeater.cc:503] Master 127.5.71.190:42921 was elected leader, sending a full tablet report...
W20260923 20:49:34.391604 18142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:34.392019 18142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:34.392206 18142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:34.400141 18142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:34.400295 18142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:34.400480 18142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:49:34.411013 18142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.71.148:56553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:42921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:34.412622 18142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:34.413935 18142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:34.428232 18147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:34.428200 18148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:34.428166 18150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:34.428949 18142 server_base.cc:1034] running on GCE node
I20260923 20:49:34.429728 18142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:34.430775 18142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:34.432098 18142 hybrid_clock.cc:648] HybridClock initialized: now 1790196574432045 us; error 51 us; skew 500 ppm
I20260923 20:49:34.432492 18142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:34 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:34.435870 18142 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:49:34.437388 18142 webserver.cc:460] Webserver started at http://127.5.71.131:45545/ using document root <none> and password file <none>
I20260923 20:49:34.438109 18142 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:34.438265 18142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:34.438597 18142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 20:49:34 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:34.454536 18142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "0d2521214fea4f5ab3805ae0c7c9a90e"
format_stamp: "Formatted at 2026-09-23 20:49:34 on dist-test-slave-dn29"
server_key: "70ff077ebdf853d979132e78d533ef55"
server_key_iv: "8b20cdde0d82e31dbe52caf485b424bb"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:34.455441 18142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "0d2521214fea4f5ab3805ae0c7c9a90e"
format_stamp: "Formatted at 2026-09-23 20:49:34 on dist-test-slave-dn29"
server_key: "70ff077ebdf853d979132e78d533ef55"
server_key_iv: "8b20cdde0d82e31dbe52caf485b424bb"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:34.461619 18142 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260923 20:49:34.477958 18157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:34.480005 18142 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 20:49:34.480225 18142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "0d2521214fea4f5ab3805ae0c7c9a90e"
format_stamp: "Formatted at 2026-09-23 20:49:34 on dist-test-slave-dn29"
server_key: "70ff077ebdf853d979132e78d533ef55"
server_key_iv: "8b20cdde0d82e31dbe52caf485b424bb"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:34.480516 18142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:34.503068 18142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:34.507885 18142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:34.508159 18142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:34.509189 18142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:34.510643 18142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:34.510776 18142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:34.510958 18142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:34.511045 18142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:34.553035 18142 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:39215
I20260923 20:49:34.553074 18269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:39215 every 8 connection(s)
I20260923 20:49:34.554391 18142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:49:34.561623 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18142
I20260923 20:49:34.561836 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:49:34.562459 5406 external_mini_cluster.cc:1428] Setting key 5ad52d5497d279f353390452ff19c57f
Sep 23 20:49:34 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196574, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:34.576220 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:34.557041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:38651 (local address 127.5.71.190:42921)
0923 20:49:34.557545 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:34.557550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:34.558739 (+ 1189us) server_negotiation.cc:407] Connection header received
0923 20:49:34.560595 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:34.560600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:34.560716 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:34.560837 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:34.562411 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.563436 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.565373 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.565911 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.566047 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:34.569674 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:34.569679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:34.569682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:34.569720 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:34.572566 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.573433 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.573437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.573440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.573564 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.574121 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.574124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.574126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.574456 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:34.574662 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:34.575527 (+ 865us) server_negotiation.cc:299] Negotiation successful
0923 20:49:34.575711 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":275,"threads_started":1}
I20260923 20:49:34.577346 18273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:34.557632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42921 (local address 127.5.71.131:38651)
0923 20:49:34.558520 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:34.558563 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:34.560308 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:34.561054 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:34.561062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:34.561502 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:34.562230 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:34.562249 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.563630 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.563635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:34.565178 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:34.565206 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.566155 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.566163 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:34.566415 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:34.567056 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:34.567080 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:34.569417 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:34.572769 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:34.572777 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:34.572791 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:34.573284 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:34.573725 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:34.573731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:34.573733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:34.573983 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:34.574752 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:34.574759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:34.575124 (+ 365us) client_negotiation.cc:769] Sending connection context
0923 20:49:34.575382 (+ 258us) client_negotiation.cc:240] Negotiation successful
0923 20:49:34.575893 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":369,"threads_started":1}
I20260923 20:49:34.579444 18270 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42921
I20260923 20:49:34.579879 18270 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:34.580680 18270 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
I20260923 20:49:34.582397 17779 ts_manager.cc:194] Registered new tserver with Master: 0d2521214fea4f5ab3805ae0c7c9a90e (127.5.71.131:39215)
I20260923 20:49:34.583668 17779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:38651
I20260923 20:49:34.591655 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:49:34.593475 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 17860
Sep 23 20:49:34 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:49:34.612694 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:34.594626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.5.71.129:44237)
0923 20:49:34.595429 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:34.595437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:34.595465 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:49:34.595523 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:34.595529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:34.595770 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:34.596028 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:34.597615 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.598824 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.600417 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.601119 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.601242 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:34.606256 (+ 5014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:34.606272 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:34.606292 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:34.606336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:34.609425 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.610307 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.610315 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.610318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.610466 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.610901 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.610906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.610909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.611335 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:34.611673 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:34.612029 (+ 356us) server_negotiation.cc:299] Negotiation successful
0923 20:49:34.612317 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":400,"threads_started":1}
I20260923 20:49:34.614048 17988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:34.698688 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 17860
I20260923 20:49:34.714788 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 17999
Sep 23 20:49:34 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:49:34.738407 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:34.715776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44952 (local address 127.5.71.130:38139)
0923 20:49:34.716396 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:34.716404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:34.716431 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:49:34.716491 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:34.716497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:34.716792 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:34.716924 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:34.718372 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.719460 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.721185 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.721869 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.722003 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:34.731989 (+ 9986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:34.732008 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:34.732027 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:34.732111 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:34.735357 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.736254 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.736261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.736264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.736402 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.736854 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.736857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.736859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.737234 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:34.737459 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:34.737911 (+ 452us) server_negotiation.cc:299] Negotiation successful
0923 20:49:34.738116 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":320,"threads_started":1}
I20260923 20:49:34.740130 18131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:34.824227 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 17999
I20260923 20:49:34.839957 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18142
Sep 23 20:49:34 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:49:34.859234 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:34.840997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57464 (local address 127.5.71.131:39215)
0923 20:49:34.841608 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:34.841615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:34.841644 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:49:34.841760 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:34.841766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:34.841997 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:34.842131 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:34.843491 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.844752 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.846319 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:34.846897 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:34.847019 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:34.852602 (+ 5583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:34.852624 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:34.852650 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:34.852708 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:34.856302 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.857114 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.857122 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.857124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.857272 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:34.857690 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:34.857694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:34.857697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:34.858040 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:34.858322 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:34.858587 (+ 265us) server_negotiation.cc:299] Negotiation successful
0923 20:49:34.858803 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":319,"threads_started":1}
I20260923 20:49:34.860626 18265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:34.946203 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18142
I20260923 20:49:34.962775 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 17748
I20260923 20:49:34.964239 17810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:35.075140 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 17748
I20260923 20:49:35.099341 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42921
--webserver_interface=127.5.71.190
--webserver_port=33949
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:42921
--ranger_config_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:35.436703 18282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:35.437108 18282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:35.437296 18282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:35.445755 18282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:35.445933 18282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:35.446043 18282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:35.446146 18282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:35.446239 18282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:35.456878 18282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:42921
--ranger_config_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:42921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=33949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:35.458362 18282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:35.459766 18282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:35.472555 18290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:35.472586 18287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:35.472616 18288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:35.474243 18282 server_base.cc:1034] running on GCE node
I20260923 20:49:35.475222 18282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:35.476852 18282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:35.478137 18282 hybrid_clock.cc:648] HybridClock initialized: now 1790196575478050 us; error 46 us; skew 500 ppm
I20260923 20:49:35.478596 18282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:35 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196575, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:35.482033 18282 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:35.483777 18282 webserver.cc:460] Webserver started at http://127.5.71.190:33949/ using document root <none> and password file <none>
I20260923 20:49:35.484555 18282 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:35.484709 18282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 20:49:35 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196575, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:35.542407 18282 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:49:35.546329 18298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:35.548018 18282 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:49:35.548240 18282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "8bcd15417eac444faedbce35875821e2"
format_stamp: "Formatted at 2026-09-23 20:49:31 on dist-test-slave-dn29"
server_key: "1c2822b8180e8baee16da7e299c42777"
server_key_iv: "625d78984ee7aabc8a33fc1f0ef53644"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:35.549310 18282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:35.577605 18282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:35.582226 18282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:35.582688 18282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:35.605329 18282 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:42921
I20260923 20:49:35.605366 18349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:42921 every 8 connection(s)
I20260923 20:49:35.606976 18282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:35.615275 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18282
I20260923 20:49:35.615656 18350 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:49:35.616389 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:44237
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:42921
--webserver_port=40701
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:35.619652 18350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Bootstrap starting.
I20260923 20:49:35.647053 18350 log.cc:824] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:35.659977 18350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:49:35.660633 18350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Bootstrap complete.
I20260923 20:49:35.673931 18350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:35.675243 18350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bcd15417eac444faedbce35875821e2, State: Initialized, Role: FOLLOWER
I20260923 20:49:35.676074 18350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:35.676363 18350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:49:35.676492 18350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:49:35.676829 18350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 20:49:35.682715 18350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:35.683390 18350 leader_election.cc:302] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bcd15417eac444faedbce35875821e2; no voters:
I20260923 20:49:35.684204 18350 leader_election.cc:288] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 20:49:35.684360 18354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 20:49:35.686733 18354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [term 2 LEADER]: Becoming Leader. State: Replica: 8bcd15417eac444faedbce35875821e2, State: Running, Role: LEADER
I20260923 20:49:35.687451 18350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:35.687322 18354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } }
I20260923 20:49:35.689980 18356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bcd15417eac444faedbce35875821e2. Latest consensus state: current_term: 2 leader_uuid: "8bcd15417eac444faedbce35875821e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } } }
I20260923 20:49:35.690385 18356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:35.690783 18355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8bcd15417eac444faedbce35875821e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd15417eac444faedbce35875821e2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 42921 } } }
I20260923 20:49:35.691146 18355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:35.701010 18363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:35.707271 18363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:35.708884 18363 catalog_manager.cc:1259] Loaded cluster ID: 03c4093272274f2298c8979978f4e605
I20260923 20:49:35.708987 18363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:49:35.713150 18363 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:49:35.714850 18363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Loaded TSK: 0
I20260923 20:49:35.716866 18363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:49:35.720247 18350 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260923 20:49:36.125452 18352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:36.126125 18352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:36.126395 18352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:36.141158 18352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:36.141419 18352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:36.141685 18352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:49:36.160563 18352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:44237
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=40701
--tserver_master_addrs=127.5.71.190:42921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:36.162811 18352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:36.164891 18352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:36.196175 18387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:36.201895 18388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:36.210165 18391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:36.211367 18352 server_base.cc:1034] running on GCE node
I20260923 20:49:36.212579 18352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:36.214205 18352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:36.215745 18352 hybrid_clock.cc:648] HybridClock initialized: now 1790196576215689 us; error 43 us; skew 500 ppm
I20260923 20:49:36.216265 18352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:36 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196576, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:36.228364 18352 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:49:36.231870 18352 webserver.cc:460] Webserver started at http://127.5.71.129:40701/ using document root <none> and password file <none>
I20260923 20:49:36.232954 18352 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:36.233139 18352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 20:49:36 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196576, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:36.275821 18352 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.007s sys 0.001s
I20260923 20:49:36.279901 18398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:36.282080 18352 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:49:36.282367 18352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "d05f60f7aba143489f8df618141ae0eb"
format_stamp: "Formatted at 2026-09-23 20:49:32 on dist-test-slave-dn29"
server_key: "2abb5f3082967eb783163b74072cd92a"
server_key_iv: "2fb0e51cad145e8778879b32c0fa6bfa"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:36.284036 18352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:36.336529 18352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:36.341730 18352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:36.342078 18352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:36.343410 18352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:36.346047 18352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:36.346197 18352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:36.346333 18352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:36.346453 18352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:36.419905 18352 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:44237
I20260923 20:49:36.419986 18511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:44237 every 8 connection(s)
I20260923 20:49:36.421761 18352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:49:36.422457 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18352
I20260923 20:49:36.423599 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:38139
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:42921
--webserver_port=44581
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:49:36 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196576, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:36.467620 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:36.425765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42921 (local address 127.5.71.129:52781)
0923 20:49:36.437267 (+ 11502us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:36.437333 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:36.440498 (+ 3165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:36.442787 (+ 2289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:36.442800 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:36.443601 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:36.444552 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:36.444583 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:36.447400 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:36.447408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:36.449608 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:36.449638 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:36.451680 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:36.451689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:36.452043 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:36.452944 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:36.452980 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:36.456595 (+ 3615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:36.461697 (+ 5102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:36.461708 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:36.461740 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:36.462446 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:36.463078 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:36.463083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:36.463087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:36.463362 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:36.464596 (+ 1234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:36.464605 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:36.465143 (+ 538us) client_negotiation.cc:769] Sending connection context
0923 20:49:36.465474 (+ 331us) client_negotiation.cc:240] Negotiation successful
0923 20:49:36.466118 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11115,"thread_start_us":10889,"threads_started":1}
I20260923 20:49:36.467705 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:36.425666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:52781 (local address 127.5.71.190:42921)
0923 20:49:36.436512 (+ 10846us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:36.436540 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:36.437886 (+ 1346us) server_negotiation.cc:407] Connection header received
0923 20:49:36.441350 (+ 3464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:36.441375 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:36.441979 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:36.442505 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:36.445101 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:36.447146 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:36.449863 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:36.450976 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:36.451217 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:36.456990 (+ 5773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:36.457004 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:36.457028 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:36.457093 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:36.461413 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:36.462659 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:36.462669 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:36.462679 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:36.462916 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:36.463511 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:36.463516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:36.463519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:36.464284 (+ 765us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:36.464511 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:36.465850 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0923 20:49:36.466338 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10421,"thread_start_us":12043,"threads_started":1}
I20260923 20:49:36.472420 18512 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42921
I20260923 20:49:36.472811 18512 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:36.473834 18512 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
W20260923 20:49:36.477756 18512 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:42921 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 20:49:36.478032 18512 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:36.478372 18512 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
I20260923 20:49:36.479933 18512 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:36.480252 18512 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
W20260923 20:49:36.481598 18512 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 20:49:36.872977 18516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:36.873329 18516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:36.873497 18516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:36.881940 18516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:36.882146 18516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:36.882340 18516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:49:36.894369 18516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:38139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=44581
--tserver_master_addrs=127.5.71.190:42921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:36.895996 18516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:36.897424 18516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:36.914178 18531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:36.915006 18516 server_base.cc:1034] running on GCE node
W20260923 20:49:36.916064 18530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:36.916430 18533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:36.917150 18516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:36.918279 18516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:36.919554 18516 hybrid_clock.cc:648] HybridClock initialized: now 1790196576919482 us; error 68 us; skew 500 ppm
I20260923 20:49:36.920208 18516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:36 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196576, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:36.924465 18516 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:49:36.926285 18516 webserver.cc:460] Webserver started at http://127.5.71.130:44581/ using document root <none> and password file <none>
I20260923 20:49:36.927064 18516 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:36.927229 18516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 20:49:36 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196576, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:36.969096 18516 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260923 20:49:36.972808 18543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:36.974790 18516 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:49:36.975054 18516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "6c60a7580673462f86fda9114ceafac6"
format_stamp: "Formatted at 2026-09-23 20:49:33 on dist-test-slave-dn29"
server_key: "97c5e0cd61db2ba75ee1506fb3af1dbb"
server_key_iv: "a08a565de3291ae74f07bd1f36acaedd"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:36.976615 18516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:37.010389 18516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:37.016587 18516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:37.017015 18516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:37.018466 18516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:37.020507 18516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:37.020691 18516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260923 20:49:37.020861 18516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:37.020968 18516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790196577, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:37.077203 18516 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38139
I20260923 20:49:37.077275 18655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38139 every 8 connection(s)
I20260923 20:49:37.079286 18516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:49:37.083721 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18516
I20260923 20:49:37.084542 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.71.148:56553/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:39215
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:42921
--webserver_port=45545
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196576, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:37.128476 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.083063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42921 (local address 127.5.71.130:39507)
0923 20:49:37.095601 (+ 12538us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:37.095659 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:37.098532 (+ 2873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:37.099880 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:37.099897 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:37.100620 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:37.101621 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:37.101654 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.103592 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.103598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:37.106040 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:37.106058 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.108628 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.108637 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:37.108991 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:37.109905 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:37.109954 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:37.113201 (+ 3247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:37.123322 (+ 10121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:37.123337 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:37.123361 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:37.124087 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:37.124588 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:37.124593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:37.124597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:37.124890 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:37.125610 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:37.125620 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:37.126071 (+ 451us) client_negotiation.cc:769] Sending connection context
0923 20:49:37.126387 (+ 316us) client_negotiation.cc:240] Negotiation successful
0923 20:49:37.126985 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12184,"thread_start_us":11945,"threads_started":1}
I20260923 20:49:37.130363 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.083761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:39507 (local address 127.5.71.190:42921)
0923 20:49:37.094949 (+ 11188us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:37.094959 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:37.098972 (+ 4013us) server_negotiation.cc:407] Connection header received
0923 20:49:37.099054 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:37.099062 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:37.099169 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:37.099296 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:37.101935 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.103359 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.106279 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.108066 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.108244 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:37.113545 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:37.113553 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:37.113559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:37.113621 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:37.123061 (+ 9440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.124299 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.124305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.124309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.124463 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.125053 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.125058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.125062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.125461 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:37.129615 (+ 4154us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:37.129776 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 20:49:37.130018 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11011,"thread_start_us":10861,"threads_started":1}
I20260923 20:49:37.132017 18656 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42921
I20260923 20:49:37.132341 18656 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:37.133195 18656 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
I20260923 20:49:37.135757 18315 ts_manager.cc:194] Registered new tserver with Master: 6c60a7580673462f86fda9114ceafac6 (127.5.71.130:38139)
I20260923 20:49:37.137979 18315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:39507
I20260923 20:49:37.481776 18512 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:37.482245 18512 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
I20260923 20:49:37.483873 18315 ts_manager.cc:194] Registered new tserver with Master: d05f60f7aba143489f8df618141ae0eb (127.5.71.129:44237)
I20260923 20:49:37.485044 18315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:52781
W20260923 20:49:37.513031 18660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:37.513573 18660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:37.513773 18660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:37.524634 18660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:37.524863 18660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:37.525117 18660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:49:37.538777 18660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.71.148:56553/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:39215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=45545
--tserver_master_addrs=127.5.71.190:42921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:37.540483 18660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:37.541908 18660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:37.561057 18666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:37.562939 18667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:37.570240 18669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:37.571555 18660 server_base.cc:1034] running on GCE node
I20260923 20:49:37.572395 18660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:37.573594 18660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:37.574920 18660 hybrid_clock.cc:648] HybridClock initialized: now 1790196577574853 us; error 53 us; skew 500 ppm
I20260923 20:49:37.575480 18660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196577, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:37.582935 18660 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:49:37.600611 18660 webserver.cc:460] Webserver started at http://127.5.71.131:45545/ using document root <none> and password file <none>
I20260923 20:49:37.601754 18660 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:37.601946 18660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196577, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196577, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:37.674970 18660 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.006s sys 0.005s
I20260923 20:49:37.690780 18677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:37.697249 18660 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.004s sys 0.010s
I20260923 20:49:37.697634 18660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "0d2521214fea4f5ab3805ae0c7c9a90e"
format_stamp: "Formatted at 2026-09-23 20:49:34 on dist-test-slave-dn29"
server_key: "70ff077ebdf853d979132e78d533ef55"
server_key_iv: "8b20cdde0d82e31dbe52caf485b424bb"
server_key_version: "kuduclusterkey@0"
I20260923 20:49:37.699242 18660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:37.741209 18660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:37.747431 18660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:37.747875 18660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:37.755425 18660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:37.764062 18660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:37.764300 18660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:37.764465 18660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:37.764581 18660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:37.834052 18660 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:39215
I20260923 20:49:37.834108 18789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:39215 every 8 connection(s)
I20260923 20:49:37.835615 18660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:49:37.838289 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18660
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196577, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:37.885484 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.840548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:49625 (local address 127.5.71.190:42921)
0923 20:49:37.849253 (+ 8705us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:37.849259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:37.853276 (+ 4017us) server_negotiation.cc:407] Connection header received
0923 20:49:37.860689 (+ 7413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:37.860696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:37.860821 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:37.860908 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:37.863474 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.864827 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.867497 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.868230 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.868331 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:37.875104 (+ 6773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:37.875112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:37.875116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:37.875161 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:37.880214 (+ 5053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.881372 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.881378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.881382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.881542 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.882140 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.882147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.882151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.882526 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:37.882646 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:37.884784 (+ 2138us) server_negotiation.cc:299] Negotiation successful
0923 20:49:37.885049 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8600,"thread_start_us":8463,"threads_started":1}
I20260923 20:49:37.886721 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.841412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:42921 (local address 127.5.71.131:49625)
0923 20:49:37.853048 (+ 11636us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:37.853122 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:37.857026 (+ 3904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:37.861576 (+ 4550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:37.861591 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:37.862276 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:37.863207 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:37.863236 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.865105 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.865110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:37.867305 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:37.867321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.868744 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.868752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:37.869079 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:37.869923 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:37.869965 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:37.873845 (+ 3880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:37.880429 (+ 6584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:37.880442 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:37.880464 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:37.881206 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:37.881708 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:37.881712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:37.881716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:37.881987 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:37.883609 (+ 1622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:37.883618 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:37.884187 (+ 569us) client_negotiation.cc:769] Sending connection context
0923 20:49:37.884582 (+ 395us) client_negotiation.cc:240] Negotiation successful
0923 20:49:37.885197 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11235,"thread_start_us":10942,"threads_started":1}
I20260923 20:49:37.888765 18790 heartbeater.cc:347] Connected to a master server at 127.5.71.190:42921
I20260923 20:49:37.889153 18790 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:37.890192 18790 heartbeater.cc:511] Master 127.5.71.190:42921 requested a full tablet report, sending...
I20260923 20:49:37.892136 18315 ts_manager.cc:194] Registered new tserver with Master: 0d2521214fea4f5ab3805ae0c7c9a90e (127.5.71.131:39215)
I20260923 20:49:37.893285 18315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:49625
I20260923 20:49:37.894682 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.840178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51080 (local address 127.5.71.190:42921)
0923 20:49:37.848825 (+ 8647us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:37.848836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:37.860056 (+ 11220us) server_negotiation.cc:407] Connection header received
0923 20:49:37.860564 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:37.860571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:37.860691 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:37.860778 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:37.863221 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.868028 (+ 4807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.870270 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.870950 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.871041 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:37.877192 (+ 6151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:37.877200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:37.877205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:37.877272 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:37.882177 (+ 4905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.883635 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.883642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.883646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.883850 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.888845 (+ 4995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.888852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.888856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.889418 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:37.889709 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:37.894012 (+ 4303us) server_negotiation.cc:299] Negotiation successful
0923 20:49:37.894271 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":8429,"thread_start_us":8288,"threads_started":1}
I20260923 20:49:37.897820 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:49:37.926393 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.901541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.5.71.190:42921)
0923 20:49:37.901888 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:37.901894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:37.908065 (+ 6171us) server_negotiation.cc:407] Connection header received
0923 20:49:37.908377 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:37.908384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:37.908499 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:37.908605 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:37.911046 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.912382 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.914647 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:37.915303 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:37.915467 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:37.919222 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:37.919229 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:37.919234 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:37.919282 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:37.922984 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.923948 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.923954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.923958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.924111 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:37.924557 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:37.924562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:37.924565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:37.924956 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:37.925197 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:37.925802 (+ 605us) server_negotiation.cc:299] Negotiation successful
0923 20:49:37.926047 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260923 20:49:37.933982 18315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:49:37.944559 18315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 23 20:49:37 dist-test-slave-dn29 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196577, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for HTTP/127.5.71.148@KRBTEST.COM
I20260923 20:49:38.013129 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.987092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:39215 (local address 127.0.0.1:57468)
0923 20:49:38.001998 (+ 14906us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:38.002018 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:38.002140 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:38.003338 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:38.003346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:38.003394 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:38.004073 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.004094 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.006913 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.006919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.009779 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.009801 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.012011 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.012019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.012278 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.012350 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 20:49:38.012506 (+ 156us) client_negotiation.cc:240] Negotiation successful
0923 20:49:38.012730 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14784,"thread_start_us":13412,"threads_started":1}
I20260923 20:49:38.014397 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.987844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.5.71.131:39215)
0923 20:49:38.001578 (+ 13734us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.001587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.002263 (+ 676us) server_negotiation.cc:407] Connection header received
0923 20:49:38.002383 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.002392 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.002752 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.002967 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:38.004375 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.006696 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.009998 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.011631 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.013424 (+ 1793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.013754 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.013838 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.014030 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11045,"thread_start_us":10872,"threads_started":1}
I20260923 20:49:38.019248 18725 tablet_service.cc:1467] Processing CreateTablet for tablet ba91fd8c8a364fc1b8da8d894f7753d6 (DEFAULT_TABLE table=test-table [id=c2448a2489dd4268bb1dc65f05ec33a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:38.021708 18725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba91fd8c8a364fc1b8da8d894f7753d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:38.026510 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.982881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44237 (local address 127.0.0.1:49620)
0923 20:49:38.000971 (+ 18090us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:38.001013 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:38.001227 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:38.004463 (+ 3236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:38.004469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:38.004507 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:38.004892 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.004906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.016517 (+ 11611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.016523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.018633 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.018649 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.025516 (+ 6867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.025528 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.025752 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.025791 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 20:49:38.025901 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 20:49:38.026086 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17781,"mutex_wait_us":24,"thread_start_us":19495,"threads_started":1}
I20260923 20:49:38.028261 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.003216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.5.71.130:38139)
0923 20:49:38.011137 (+ 7921us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.011147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.011176 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:49:38.011272 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.011279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.011517 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.011770 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:38.012424 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.013635 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.015846 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.017348 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.017483 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.017693 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.027681 (+ 9988us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.027923 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7769,"thread_start_us":7635,"threads_started":1}
I20260923 20:49:38.029060 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:37.983178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.5.71.129:44237)
0923 20:49:38.002298 (+ 19120us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.002309 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.002342 (+ 33us) server_negotiation.cc:407] Connection header received
0923 20:49:38.002457 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.002465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.002755 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.005137 (+ 2382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:38.005470 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.007222 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.018855 (+ 11633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.024640 (+ 5785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.025343 (+ 703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.025605 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.027911 (+ 2306us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.028126 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18914,"thread_start_us":19735,"threads_started":1}
I20260923 20:49:38.027501 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.006502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38139 (local address 127.0.0.1:44966)
0923 20:49:38.010429 (+ 3927us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:38.010456 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:38.010628 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:38.011749 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:38.011754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:38.011806 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:38.012233 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.012243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.013802 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.013809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.015598 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.015617 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.017876 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.017883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.026823 (+ 8940us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.026861 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 20:49:38.026956 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 20:49:38.027119 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3752,"mutex_wait_us":92,"thread_start_us":3600,"threads_started":1}
I20260923 20:49:38.033972 18447 tablet_service.cc:1467] Processing CreateTablet for tablet ba91fd8c8a364fc1b8da8d894f7753d6 (DEFAULT_TABLE table=test-table [id=c2448a2489dd4268bb1dc65f05ec33a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:38.038166 18591 tablet_service.cc:1467] Processing CreateTablet for tablet ba91fd8c8a364fc1b8da8d894f7753d6 (DEFAULT_TABLE table=test-table [id=c2448a2489dd4268bb1dc65f05ec33a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:38.041683 18591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba91fd8c8a364fc1b8da8d894f7753d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:38.049011 18447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba91fd8c8a364fc1b8da8d894f7753d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:38.068200 18812 tablet_bootstrap.cc:492] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Bootstrap starting.
I20260923 20:49:38.096830 18812 tablet_bootstrap.cc:654] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:38.100651 18814 tablet_bootstrap.cc:492] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Bootstrap starting.
I20260923 20:49:38.103927 18812 log.cc:824] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:38.105896 18814 tablet_bootstrap.cc:654] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:38.108086 18814 log.cc:824] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:38.118006 18812 tablet_bootstrap.cc:492] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: No bootstrap required, opened a new log
I20260923 20:49:38.118463 18812 ts_tablet_manager.cc:1397] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.003s
I20260923 20:49:38.124809 18813 tablet_bootstrap.cc:492] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Bootstrap starting.
I20260923 20:49:38.129475 18814 tablet_bootstrap.cc:492] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: No bootstrap required, opened a new log
I20260923 20:49:38.129946 18814 ts_tablet_manager.cc:1397] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Time spent bootstrapping tablet: real 0.030s user 0.016s sys 0.000s
I20260923 20:49:38.135429 18813 tablet_bootstrap.cc:654] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:38.136801 18813 log.cc:824] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:38.142773 18656 heartbeater.cc:503] Master 127.5.71.190:42921 was elected leader, sending a full tablet report...
I20260923 20:49:38.142469 18814 raft_consensus.cc:348] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.144831 18814 raft_consensus.cc:374] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:38.144575 18812 raft_consensus.cc:348] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.145004 18812 raft_consensus.cc:374] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:38.145117 18812 raft_consensus.cc:729] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2521214fea4f5ab3805ae0c7c9a90e, State: Initialized, Role: FOLLOWER
I20260923 20:49:38.145265 18814 raft_consensus.cc:729] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c60a7580673462f86fda9114ceafac6, State: Initialized, Role: FOLLOWER
I20260923 20:49:38.145843 18812 consensus_queue.cc:260] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.146270 18814 consensus_queue.cc:260] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.154728 18790 heartbeater.cc:503] Master 127.5.71.190:42921 was elected leader, sending a full tablet report...
I20260923 20:49:38.155833 18812 ts_tablet_manager.cc:1428] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Time spent starting tablet: real 0.037s user 0.015s sys 0.003s
I20260923 20:49:38.157982 18813 tablet_bootstrap.cc:492] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: No bootstrap required, opened a new log
I20260923 20:49:38.158419 18813 ts_tablet_manager.cc:1397] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.000s
I20260923 20:49:38.173750 18814 ts_tablet_manager.cc:1428] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Time spent starting tablet: real 0.044s user 0.018s sys 0.005s
I20260923 20:49:38.176491 18813 raft_consensus.cc:348] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.177259 18813 raft_consensus.cc:374] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:38.177441 18813 raft_consensus.cc:729] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d05f60f7aba143489f8df618141ae0eb, State: Initialized, Role: FOLLOWER
I20260923 20:49:38.178397 18813 consensus_queue.cc:260] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.182704 18512 heartbeater.cc:503] Master 127.5.71.190:42921 was elected leader, sending a full tablet report...
I20260923 20:49:38.187361 18813 ts_tablet_manager.cc:1428] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Time spent starting tablet: real 0.029s user 0.016s sys 0.003s
W20260923 20:49:38.334295 18657 tablet.cc:2369] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:49:38.361094 18791 tablet.cc:2369] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:49:38.431185 18513 tablet.cc:2369] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:38.437325 18820 raft_consensus.cc:482] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:38.437696 18820 raft_consensus.cc:504] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.439582 18820 leader_election.cc:288] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d2521214fea4f5ab3805ae0c7c9a90e (127.5.71.131:39215), 6c60a7580673462f86fda9114ceafac6 (127.5.71.130:38139)
I20260923 20:49:38.449522 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.439994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:38559 (local address 127.5.71.131:39215)
0923 20:49:38.440479 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.440490 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.441234 (+ 744us) server_negotiation.cc:407] Connection header received
0923 20:49:38.441460 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.441464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.441520 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.441597 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:38.442328 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.443334 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.446596 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.447958 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.448131 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.448209 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.449101 (+ 892us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.449248 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260923 20:49:38.449522 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.440195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:46855 (local address 127.5.71.130:38139)
0923 20:49:38.440650 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.440656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.440811 (+ 155us) server_negotiation.cc:407] Connection header received
0923 20:49:38.441069 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.441073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.441150 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.441274 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:38.441995 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.443093 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.446596 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.447958 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.448130 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.448270 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.449101 (+ 831us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.449258 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291}
I20260923 20:49:38.449800 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.439873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38139 (local address 127.5.71.129:46855)
0923 20:49:38.440669 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:38.440693 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:38.440908 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:38.441316 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:38.441321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:38.441351 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:38.441811 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.441826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.443284 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.443288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.446357 (+ 3069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.446377 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.448291 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.448313 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.448750 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.448878 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 20:49:38.448996 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 20:49:38.449213 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":197,"thread_start_us":393,"threads_started":1}
I20260923 20:49:38.450177 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.440057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:39215 (local address 127.5.71.129:38559)
0923 20:49:38.441105 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:38.441131 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:38.441301 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:38.441658 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:38.441663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:38.441690 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:38.442131 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.442140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.443473 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.443478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.446368 (+ 2890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:38.446377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.448313 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.448320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:38.448749 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.448882 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 20:49:38.448996 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 20:49:38.449213 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":399,"thread_start_us":684,"threads_started":1}
I20260923 20:49:38.451072 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba91fd8c8a364fc1b8da8d894f7753d6" candidate_uuid: "d05f60f7aba143489f8df618141ae0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c60a7580673462f86fda9114ceafac6" is_pre_election: true
I20260923 20:49:38.451225 18745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba91fd8c8a364fc1b8da8d894f7753d6" candidate_uuid: "d05f60f7aba143489f8df618141ae0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" is_pre_election: true
I20260923 20:49:38.451651 18611 raft_consensus.cc:2454] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d05f60f7aba143489f8df618141ae0eb in term 0.
I20260923 20:49:38.451887 18745 raft_consensus.cc:2454] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d05f60f7aba143489f8df618141ae0eb in term 0.
I20260923 20:49:38.452432 18400 leader_election.cc:302] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c60a7580673462f86fda9114ceafac6, d05f60f7aba143489f8df618141ae0eb; no voters:
I20260923 20:49:38.452930 18820 raft_consensus.cc:2781] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:49:38.453091 18820 raft_consensus.cc:482] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:49:38.453190 18820 raft_consensus.cc:3037] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:38.455474 18820 raft_consensus.cc:504] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.456179 18820 leader_election.cc:288] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [CANDIDATE]: Term 1 election: Requested vote from peers 0d2521214fea4f5ab3805ae0c7c9a90e (127.5.71.131:39215), 6c60a7580673462f86fda9114ceafac6 (127.5.71.130:38139)
I20260923 20:49:38.456800 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba91fd8c8a364fc1b8da8d894f7753d6" candidate_uuid: "d05f60f7aba143489f8df618141ae0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c60a7580673462f86fda9114ceafac6"
I20260923 20:49:38.456846 18745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba91fd8c8a364fc1b8da8d894f7753d6" candidate_uuid: "d05f60f7aba143489f8df618141ae0eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e"
I20260923 20:49:38.457052 18611 raft_consensus.cc:3037] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:38.457120 18745 raft_consensus.cc:3037] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:38.459396 18745 raft_consensus.cc:2454] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d05f60f7aba143489f8df618141ae0eb in term 1.
I20260923 20:49:38.459396 18611 raft_consensus.cc:2454] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d05f60f7aba143489f8df618141ae0eb in term 1.
I20260923 20:49:38.460011 18402 leader_election.cc:302] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d2521214fea4f5ab3805ae0c7c9a90e, d05f60f7aba143489f8df618141ae0eb; no voters:
I20260923 20:49:38.460366 18820 raft_consensus.cc:2781] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:38.460903 18820 raft_consensus.cc:686] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 1 LEADER]: Becoming Leader. State: Replica: d05f60f7aba143489f8df618141ae0eb, State: Running, Role: LEADER
I20260923 20:49:38.461427 18820 consensus_queue.cc:237] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } }
I20260923 20:49:38.466722 18315 catalog_manager.cc:5495] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb reported cstate change: term changed from 0 to 1, leader changed from <none> to d05f60f7aba143489f8df618141ae0eb (127.5.71.129). New cstate: current_term: 1 leader_uuid: "d05f60f7aba143489f8df618141ae0eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05f60f7aba143489f8df618141ae0eb" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 } health_report { overall_health: UNKNOWN } } }
I20260923 20:49:38.568987 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.562370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49622 (local address 127.5.71.129:44237)
0923 20:49:38.563215 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.563220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.563235 (+ 15us) server_negotiation.cc:407] Connection header received
0923 20:49:38.563325 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.563329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.563407 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.563565 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:49:38.564001 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.564744 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.566354 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.566801 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.566909 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.567494 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:49:38.568321 (+ 827us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:49:38.568437 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.568628 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.568740 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":592,"threads_started":1}
I20260923 20:49:38.577711 18611 raft_consensus.cc:1261] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 1 FOLLOWER]: Refusing update from remote peer d05f60f7aba143489f8df618141ae0eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:38.577760 18745 raft_consensus.cc:1261] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 1 FOLLOWER]: Refusing update from remote peer d05f60f7aba143489f8df618141ae0eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:38.578835 18830 consensus_queue.cc:1035] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d2521214fea4f5ab3805ae0c7c9a90e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 39215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:38.579236 18820 consensus_queue.cc:1035] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c60a7580673462f86fda9114ceafac6" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:38.623215 18314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:49:38.623526 18314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:49:38.627403 18314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8bcd15417eac444faedbce35875821e2: Sending DeleteTablet for 3 replicas of tablet ba91fd8c8a364fc1b8da8d894f7753d6
I20260923 20:49:38.628925 18591 tablet_service.cc:1514] Processing DeleteTablet for tablet ba91fd8c8a364fc1b8da8d894f7753d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:38 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:44966
I20260923 20:49:38.628934 18447 tablet_service.cc:1514] Processing DeleteTablet for tablet ba91fd8c8a364fc1b8da8d894f7753d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:38 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:49620
I20260923 20:49:38.629365 18725 tablet_service.cc:1514] Processing DeleteTablet for tablet ba91fd8c8a364fc1b8da8d894f7753d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:38 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57468
I20260923 20:49:38.632978 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18352
I20260923 20:49:38.642735 18846 tablet_replica.cc:330] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: stopping tablet replica
I20260923 20:49:38.643334 18846 raft_consensus.cc:2229] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:49:38.644265 18846 raft_consensus.cc:2258] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:38.644596 18845 tablet_replica.cc:330] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: stopping tablet replica
I20260923 20:49:38.645040 18845 raft_consensus.cc:2229] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:49:38.645597 18845 raft_consensus.cc:2258] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:38.649225 18846 ts_tablet_manager.cc:1905] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:38.649389 18845 ts_tablet_manager.cc:1905] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:38.649423 18847 tablet_replica.cc:330] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: stopping tablet replica
I20260923 20:49:38.649926 18847 raft_consensus.cc:2229] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:49:38.650545 18847 raft_consensus.cc:2258] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:38.656301 18847 ts_tablet_manager.cc:1905] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:38.657173 18845 ts_tablet_manager.cc:1918] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:38.657361 18845 log.cc:1196] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/ba91fd8c8a364fc1b8da8d894f7753d6
I20260923 20:49:38.657955 18845 ts_tablet_manager.cc:1939] T ba91fd8c8a364fc1b8da8d894f7753d6 P 6c60a7580673462f86fda9114ceafac6: Deleting consensus metadata
I20260923 20:49:38.658301 18846 ts_tablet_manager.cc:1918] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:38.658483 18846 log.cc:1196] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/ba91fd8c8a364fc1b8da8d894f7753d6
I20260923 20:49:38.659058 18846 ts_tablet_manager.cc:1939] T ba91fd8c8a364fc1b8da8d894f7753d6 P d05f60f7aba143489f8df618141ae0eb: Deleting consensus metadata
I20260923 20:49:38.659986 18300 catalog_manager.cc:4841] TS 6c60a7580673462f86fda9114ceafac6 (127.5.71.130:38139): tablet ba91fd8c8a364fc1b8da8d894f7753d6 (table test-table [id=c2448a2489dd4268bb1dc65f05ec33a2]) successfully deleted
I20260923 20:49:38.660936 18300 catalog_manager.cc:4841] TS d05f60f7aba143489f8df618141ae0eb (127.5.71.129:44237): tablet ba91fd8c8a364fc1b8da8d894f7753d6 (table test-table [id=c2448a2489dd4268bb1dc65f05ec33a2]) successfully deleted
I20260923 20:49:38.665632 18847 ts_tablet_manager.cc:1918] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:38.665901 18847 log.cc:1196] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/ba91fd8c8a364fc1b8da8d894f7753d6
I20260923 20:49:38.666384 18847 ts_tablet_manager.cc:1939] T ba91fd8c8a364fc1b8da8d894f7753d6 P 0d2521214fea4f5ab3805ae0c7c9a90e: Deleting consensus metadata
I20260923 20:49:38.667989 18302 catalog_manager.cc:4841] TS 0d2521214fea4f5ab3805ae0c7c9a90e (127.5.71.131:39215): tablet ba91fd8c8a364fc1b8da8d894f7753d6 (table test-table [id=c2448a2489dd4268bb1dc65f05ec33a2]) successfully deleted
I20260923 20:49:38.671505 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.634701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.5.71.129:44237)
0923 20:49:38.634964 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.634975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.644262 (+ 9287us) server_negotiation.cc:407] Connection header received
0923 20:49:38.645178 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.645187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.645304 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.645412 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:38.651093 (+ 5681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.652465 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.657441 (+ 4976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.658251 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.658363 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.663308 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:38.663323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:38.663345 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:38.663399 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:38.668100 (+ 4701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:38.668915 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:38.668926 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:38.668931 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:38.669111 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:38.669564 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:38.669570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:38.669574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:38.670092 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:38.670444 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.670772 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.671045 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 20:49:38.672472 18507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:38.776539 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18352
I20260923 20:49:38.794938 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18516
I20260923 20:49:38.810560 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.795873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.5.71.130:38139)
0923 20:49:38.796044 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.796049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.796067 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:49:38.796133 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.796137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.796208 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.796316 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:38.797852 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.798599 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.800259 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.800750 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.800859 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.803616 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:38.803641 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:38.803720 (+ 79us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:38.803799 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:38.807379 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:38.808319 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:38.808334 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:38.808336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:38.808473 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:38.808850 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:38.808853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:38.808855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:38.809264 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:38.809488 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.810024 (+ 536us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.810242 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 20:49:38.811342 18651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:38.908005 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18516
I20260923 20:49:38.924989 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18660
I20260923 20:49:38.940412 18809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:38.925921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.5.71.131:39215)
0923 20:49:38.926092 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:38.926098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:38.926155 (+ 57us) server_negotiation.cc:407] Connection header received
0923 20:49:38.926401 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:38.926405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:38.926492 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:38.926607 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:38.928116 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.928913 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.930336 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:38.930780 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:38.930925 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:38.933873 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:38.933893 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:38.933912 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:38.933948 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:38.937026 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:38.938024 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:38.938033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:38.938035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:38.938164 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:38.938561 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:38.938568 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:38.938572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:38.938983 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:38.939312 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:38.939638 (+ 326us) server_negotiation.cc:299] Negotiation successful
0923 20:49:38.940113 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 20:49:38.941092 18785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:39.034446 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18660
I20260923 20:49:39.052964 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18282
I20260923 20:49:39.054382 18345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:39.172072 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18282
I20260923 20:49:39.198297 5406 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 20:49:39.535542 5406 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 20:49:39.535854 5406 mini_ranger.cc:66] Stopping Ranger...
I20260923 20:49:39.882557 5406 mini_ranger.cc:68] Stopped Ranger
2026-09-23 20:49:39.882 UTC [17334] LOG: received smart shutdown request
2026-09-23 20:49:39.887 UTC [17334] LOG: background worker "logical replication launcher" (PID 17343) exited with exit code 1
2026-09-23 20:49:39.887 UTC [17338] LOG: shutting down
2026-09-23 20:49:40.129 UTC [17334] LOG: database system is shut down
2026-09-23T20:49:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59675 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:49:40 dist-test-slave-dn29 krb5kdc[18861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:49:40 dist-test-slave-dn29 krb5kdc[18861](info): set up 2 sockets
Sep 23 20:49:40 dist-test-slave-dn29 krb5kdc[18861](info): commencing operation
krb5kdc: starting...
W20260923 20:49:41.396610 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:49:41 dist-test-slave-dn29 krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:49:41.647190 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:39533
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:39533,127.5.71.189:46791,127.5.71.188:33645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:41.989841 18874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:41.990422 18874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:41.990614 18874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:41.998551 18874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:41.998905 18874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:41.999010 18874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:41.999110 18874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:41.999193 18874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:42.009892 18874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:39533,127.5.71.189:46791,127.5.71.188:33645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:42.011415 18874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:42.013262 18874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:42.026178 18879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:42.026211 18882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:42.026261 18880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:42.027302 18874 server_base.cc:1034] running on GCE node
I20260923 20:49:42.028337 18874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:42.029985 18874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:42.031240 18874 hybrid_clock.cc:648] HybridClock initialized: now 1790196582031184 us; error 53 us; skew 500 ppm
I20260923 20:49:42.031769 18874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:42 dist-test-slave-dn29 krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:42.041810 18874 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:42.043603 18874 webserver.cc:460] Webserver started at http://127.5.71.190:36647/ using document root <none> and password file <none>
I20260923 20:49:42.044401 18874 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:42.044515 18874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:42.044785 18874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:42.048154 18874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "3adaddfb83a24b7d8385e33d71808606"
format_stamp: "Formatted at 2026-09-23 20:49:42 on dist-test-slave-dn29"
server_key: "fab4c3b0bfde73f9323db201f32f171a"
server_key_iv: "51c2c8b9198ca006dbd44d8c782641b1"
server_key_version: "encryptionkey@0"
I20260923 20:49:42.048981 18874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "3adaddfb83a24b7d8385e33d71808606"
format_stamp: "Formatted at 2026-09-23 20:49:42 on dist-test-slave-dn29"
server_key: "fab4c3b0bfde73f9323db201f32f171a"
server_key_iv: "51c2c8b9198ca006dbd44d8c782641b1"
server_key_version: "encryptionkey@0"
I20260923 20:49:42.054821 18874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:49:42.059082 18889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:42.060951 18874 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:49:42.061270 18874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "3adaddfb83a24b7d8385e33d71808606"
format_stamp: "Formatted at 2026-09-23 20:49:42 on dist-test-slave-dn29"
server_key: "fab4c3b0bfde73f9323db201f32f171a"
server_key_iv: "51c2c8b9198ca006dbd44d8c782641b1"
server_key_version: "encryptionkey@0"
I20260923 20:49:42.061502 18874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:42.083434 18874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:42.089046 18874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:42.089362 18874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:42.111836 18874 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:39533
I20260923 20:49:42.111900 18940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:39533 every 8 connection(s)
I20260923 20:49:42.113343 18874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:42.117491 18941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:42.121996 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18874
I20260923 20:49:42.122236 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:49:42.122906 5406 external_mini_cluster.cc:1428] Setting key d09ee99a95f459d31817982bd9053d30
I20260923 20:49:42.124648 18941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:49:42 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.189@KRBTEST.COM" created.
I20260923 20:49:42.149470 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.128172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:39533 (local address 127.0.0.1:58434)
0923 20:49:42.129363 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.129397 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:42.131259 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:42.132192 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:42.132198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:42.132969 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:42.133318 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.133336 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.134913 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.134918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.136085 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.136098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.137065 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.137072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.137241 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.137936 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:42.137960 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:42.140607 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:42.144595 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.144607 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.144622 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.145091 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.145569 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.145575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.145579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.145773 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.146698 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:42.146708 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:42.147010 (+ 302us) client_negotiation.cc:769] Sending connection context
0923 20:49:42.147293 (+ 283us) client_negotiation.cc:240] Negotiation successful
0923 20:49:42.148049 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"mutex_wait_us":223,"spinlock_wait_cycles":17664,"thread_start_us":751,"threads_started":1}
I20260923 20:49:42.149489 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.128171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.5.71.190:39533)
0923 20:49:42.129077 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:42.129098 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:42.129526 (+ 428us) server_negotiation.cc:407] Connection header received
0923 20:49:42.131658 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:42.131664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:42.131965 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:42.132182 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:42.133526 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.134759 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.136230 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.136818 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.137022 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.140918 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:42.140929 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:42.140945 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:42.140982 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:42.144419 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.145269 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.145276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.145278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.145441 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.145899 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.145904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.145913 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.146485 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:42.146735 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:42.147428 (+ 693us) server_negotiation.cc:299] Negotiation successful
0923 20:49:42.147920 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":635,"thread_start_us":420,"threads_started":1}
I20260923 20:49:42.152877 18941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:49:42.154127 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.153732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51032)
0923 20:49:42.153905 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.154010 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260923 20:49:42.154502 18892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
I20260923 20:49:42.156224 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.155687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51044)
0923 20:49:42.155922 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.156025 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260923 20:49:42.156456 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:46791: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab.
Entry for principal kudu/127.5.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab.
I20260923 20:49:42.180006 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:46791
--webserver_interface=127.5.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab
--principal=kudu/127.5.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:39533,127.5.71.189:46791,127.5.71.188:33645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:42.205845 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } attempt: 1
I20260923 20:49:42.207137 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.206589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51048)
0923 20:49:42.206816 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.206952 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260923 20:49:42.208346 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.207905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51062)
0923 20:49:42.208066 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.208151 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 20:49:42.208606 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:46791: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
I20260923 20:49:42.276929 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } attempt: 2
I20260923 20:49:42.278517 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.277806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51078)
0923 20:49:42.278151 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.278316 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260923 20:49:42.279732 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.279245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51090)
0923 20:49:42.279419 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.279528 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260923 20:49:42.280107 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:46791: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
I20260923 20:49:42.371423 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } attempt: 3
I20260923 20:49:42.372852 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.372263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51106)
0923 20:49:42.372557 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.372673 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260923 20:49:42.373970 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.373550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51120)
0923 20:49:42.373697 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.373795 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 20:49:42.374183 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:46791: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
I20260923 20:49:42.517426 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } attempt: 4
I20260923 20:49:42.518757 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.518088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51126)
0923 20:49:42.518449 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.518575 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260923 20:49:42.520027 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.519553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51136)
0923 20:49:42.519720 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.519858 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260923 20:49:42.520283 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:46791: Network error: Client connection negotiation failed: client connection to 127.5.71.189:46791: connect: Connection refused (error 111)
W20260923 20:49:42.541232 18948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:42.541600 18948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:42.541749 18948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:42.549451 18948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:42.549634 18948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:42.549742 18948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:42.549844 18948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:42.549917 18948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:42.560671 18948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:39533,127.5.71.189:46791,127.5.71.188:33645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab
--principal=kudu/127.5.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:46791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:42.562121 18948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:42.563640 18948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:42.576630 18954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:42.576630 18956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:42.576766 18953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:42.578058 18948 server_base.cc:1034] running on GCE node
I20260923 20:49:42.579033 18948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:42.580780 18948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:42.582067 18948 hybrid_clock.cc:648] HybridClock initialized: now 1790196582582021 us; error 36 us; skew 500 ppm
I20260923 20:49:42.582458 18948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:42 dist-test-slave-dn29 krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:42.586441 18948 init.cc:377] Logged in from keytab as kudu/127.5.71.189@KRBTEST.COM (short username kudu)
I20260923 20:49:42.588222 18948 webserver.cc:460] Webserver started at http://127.5.71.189:37825/ using document root <none> and password file <none>
I20260923 20:49:42.588951 18948 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:42.589097 18948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:42.589416 18948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:42.592931 18948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data/instance:
uuid: "2a446de820ee4ca588ed36cae5453b30"
format_stamp: "Formatted at 2026-09-23 20:49:42 on dist-test-slave-dn29"
server_key: "c21109fed4aa248c7d2051b58bc42324"
server_key_iv: "7f29c4cd28db2f820a137d3bf4cb04c1"
server_key_version: "encryptionkey@0"
I20260923 20:49:42.593859 18948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal/instance:
uuid: "2a446de820ee4ca588ed36cae5453b30"
format_stamp: "Formatted at 2026-09-23 20:49:42 on dist-test-slave-dn29"
server_key: "c21109fed4aa248c7d2051b58bc42324"
server_key_iv: "7f29c4cd28db2f820a137d3bf4cb04c1"
server_key_version: "encryptionkey@0"
I20260923 20:49:42.599375 18948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:49:42.603293 18963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:42.605058 18948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:49:42.605314 18948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal
uuid: "2a446de820ee4ca588ed36cae5453b30"
format_stamp: "Formatted at 2026-09-23 20:49:42 on dist-test-slave-dn29"
server_key: "c21109fed4aa248c7d2051b58bc42324"
server_key_iv: "7f29c4cd28db2f820a137d3bf4cb04c1"
server_key_version: "encryptionkey@0"
I20260923 20:49:42.605545 18948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:42.633740 18948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:42.638360 18948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:42.638680 18948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:42.658026 18948 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.189:46791
I20260923 20:49:42.658048 19014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.189:46791 every 8 connection(s)
I20260923 20:49:42.659475 18948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
I20260923 20:49:42.663331 19015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:42.665552 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 18948
I20260923 20:49:42.665763 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-1/wal/instance
I20260923 20:49:42.666330 5406 external_mini_cluster.cc:1428] Setting key e83b23d4fe800ea6570a7b9fa1ee090e
I20260923 20:49:42.673592 19015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.71.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.188@KRBTEST.COM" created.
Sep 23 20:49:42 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:42.702009 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.676963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.5.71.190:39533)
0923 20:49:42.677445 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:42.677450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:42.678714 (+ 1264us) server_negotiation.cc:407] Connection header received
0923 20:49:42.681053 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:42.681059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:42.681174 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:42.681309 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:42.683030 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.683936 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.685831 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.686306 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.686462 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.693656 (+ 7194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:42.693668 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:42.693676 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:42.693745 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:42.697986 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.699018 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.699028 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.699034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.699211 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.699988 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.699992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.699994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.700320 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:42.700447 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:42.701505 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0923 20:49:42.701717 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":273,"threads_started":1}
I20260923 20:49:42.703557 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.677594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:39533 (local address 127.0.0.1:58440)
0923 20:49:42.678511 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.678554 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:42.680732 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:42.681602 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:42.681612 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:42.682208 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:42.682836 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.682856 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.684116 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.684120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.685654 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.685678 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.686597 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.686606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.686867 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.687994 (+ 1127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:42.688020 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:42.693313 (+ 5293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:42.698247 (+ 4934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.698258 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.698276 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.698862 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.699389 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.699393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.699395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.699827 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.700469 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:42.700480 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:42.701056 (+ 576us) client_negotiation.cc:769] Sending connection context
0923 20:49:42.701402 (+ 346us) client_negotiation.cc:240] Negotiation successful
0923 20:49:42.702024 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":429,"threads_started":1}
I20260923 20:49:42.705499 19015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:49:42 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:49:42.721397 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.706377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51146)
0923 20:49:42.706560 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.706577 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:42.706676 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:42.708080 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:42.708085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:42.708594 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:42.708736 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.708744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.710222 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.710230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.711230 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.711243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.712135 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.712140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.712398 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.713040 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:42.713057 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:42.715106 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:42.718741 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.718747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.718753 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.719119 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.719501 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.719506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.719510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.719686 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.720348 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:42.720353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:42.720499 (+ 146us) client_negotiation.cc:769] Sending connection context
0923 20:49:42.720649 (+ 150us) client_negotiation.cc:240] Negotiation successful
0923 20:49:42.720948 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":36}
I20260923 20:49:42.721443 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.706739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.5.71.189:46791)
0923 20:49:42.707443 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:42.707449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:42.707479 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:49:42.707537 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:42.707543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:42.707924 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:42.708104 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:42.708884 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.710076 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.711373 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.711994 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.712124 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.715339 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:42.715351 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:42.715369 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:42.715399 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:42.718573 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.719267 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.719275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.719277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.719387 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.719829 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.719834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.719838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.720216 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:42.720466 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:42.720804 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 20:49:42.721059 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":542,"thread_start_us":403,"threads_started":1}
I20260923 20:49:42.724516 19015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:49:42.725749 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.725265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54660)
0923 20:49:42.725471 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.725579 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260923 20:49:42.726007 18965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab.
Entry for principal kudu/127.5.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab.
I20260923 20:49:42.728119 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.727603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54670)
0923 20:49:42.727869 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.727972 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260923 20:49:42.728404 19015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:42.728708 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:33645
--webserver_interface=127.5.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab
--principal=kudu/127.5.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:39533,127.5.71.189:46791,127.5.71.188:33645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:42.777815 19015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 1
I20260923 20:49:42.779176 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.778679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54678)
0923 20:49:42.778905 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.779012 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260923 20:49:42.780529 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.780156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54682)
0923 20:49:42.780324 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.780404 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260923 20:49:42.780827 19015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:42.819603 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } attempt: 5
Sep 23 20:49:42 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:49:42.837252 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.820430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51150)
0923 20:49:42.820671 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.820697 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:42.820852 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:42.821267 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:42.821271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:42.821666 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:42.821843 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.821852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.823295 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.823299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.824291 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:42.824305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.825488 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.825494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:42.825646 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.826333 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:42.826353 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:42.829178 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:42.833851 (+ 4673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.833857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.833861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.834413 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.835057 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:42.835062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:42.835065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:42.835250 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:42.836112 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:42.836117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:42.836310 (+ 193us) client_negotiation.cc:769] Sending connection context
0923 20:49:42.836528 (+ 218us) client_negotiation.cc:240] Negotiation successful
0923 20:49:42.836861 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260923 20:49:42.837293 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.820519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.5.71.189:46791)
0923 20:49:42.820849 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:42.820856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:42.820886 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:49:42.821019 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:42.821025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:42.821107 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:42.821223 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:42.822006 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.823132 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.824464 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:42.825313 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:42.825456 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:42.829470 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:42.829480 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:42.829489 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:42.829545 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:42.833635 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.834551 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.834560 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.834567 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.834738 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:42.835389 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:42.835397 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:42.835403 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:42.835968 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:42.836160 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:42.836712 (+ 552us) server_negotiation.cc:299] Negotiation successful
0923 20:49:42.836983 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260923 20:49:42.838670 18941 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:49:42.840022 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.839460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54684)
0923 20:49:42.839670 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.839787 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 20:49:42.841331 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.840800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54690)
0923 20:49:42.841050 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.841166 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260923 20:49:42.841579 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:42.849066 19015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 2
I20260923 20:49:42.850569 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.849879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54698)
0923 20:49:42.850224 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.850344 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260923 20:49:42.851963 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.851489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54712)
0923 20:49:42.851715 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.851830 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260923 20:49:42.852217 19015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:42.892827 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 1
I20260923 20:49:42.894124 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.893573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54718)
0923 20:49:42.893860 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.893954 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260923 20:49:42.895201 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.894848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54730)
0923 20:49:42.895010 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.895094 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260923 20:49:42.895421 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:42.943499 19015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 3
I20260923 20:49:42.944975 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.944308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54740)
0923 20:49:42.944575 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.944726 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260923 20:49:42.946316 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.945694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54748)
0923 20:49:42.946016 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.946129 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
W20260923 20:49:42.946631 19015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:42.963678 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 2
I20260923 20:49:42.965114 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.964512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54758)
0923 20:49:42.964841 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.964942 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260923 20:49:42.966161 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:42.965806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54764)
0923 20:49:42.965950 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:42.966032 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260923 20:49:42.966391 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:43.072677 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 3
I20260923 20:49:43.073880 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.073375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54776)
0923 20:49:43.073586 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.073684 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260923 20:49:43.075022 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.074612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54778)
0923 20:49:43.074807 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.074893 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260923 20:49:43.075260 18941 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
I20260923 20:49:43.089931 19015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 4
I20260923 20:49:43.091501 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.090734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54780)
0923 20:49:43.091159 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.091329 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260923 20:49:43.092860 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.092414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54782)
0923 20:49:43.092613 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.092688 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260923 20:49:43.093148 19015 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:33645: Network error: Client connection negotiation failed: client connection to 127.5.71.188:33645: connect: Connection refused (error 111)
W20260923 20:49:43.093978 19023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:43.094326 19023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:43.094512 19023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:43.103152 19023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:43.103286 19023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:43.103403 19023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:43.103499 19023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:43.103560 19023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:43.116876 19023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:39533,127.5.71.189:46791,127.5.71.188:33645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab
--principal=kudu/127.5.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:33645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:43.118677 19023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:43.120620 19023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:43.135026 19028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:43.135084 19029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:43.135215 19031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:43.136044 19023 server_base.cc:1034] running on GCE node
I20260923 20:49:43.136932 19023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:43.138551 19023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:43.139848 19023 hybrid_clock.cc:648] HybridClock initialized: now 1790196583139737 us; error 48 us; skew 500 ppm
I20260923 20:49:43.140384 19023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:43.144070 19023 init.cc:377] Logged in from keytab as kudu/127.5.71.188@KRBTEST.COM (short username kudu)
I20260923 20:49:43.146020 19023 webserver.cc:460] Webserver started at http://127.5.71.188:45151/ using document root <none> and password file <none>
I20260923 20:49:43.146867 19023 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:43.147025 19023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:43.147363 19023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:43.150959 19023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data/instance:
uuid: "45951424fb934e688e8c47df7339d663"
format_stamp: "Formatted at 2026-09-23 20:49:43 on dist-test-slave-dn29"
server_key: "5ffb1b2c09f049fb821d231e11732bb5"
server_key_iv: "a43cdf426447c699f330dde5eef62478"
server_key_version: "encryptionkey@0"
I20260923 20:49:43.152127 19023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal/instance:
uuid: "45951424fb934e688e8c47df7339d663"
format_stamp: "Formatted at 2026-09-23 20:49:43 on dist-test-slave-dn29"
server_key: "5ffb1b2c09f049fb821d231e11732bb5"
server_key_iv: "a43cdf426447c699f330dde5eef62478"
server_key_version: "encryptionkey@0"
I20260923 20:49:43.158417 19023 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:49:43.162460 19038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:43.164289 19023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 20:49:43.164552 19023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal
uuid: "45951424fb934e688e8c47df7339d663"
format_stamp: "Formatted at 2026-09-23 20:49:43 on dist-test-slave-dn29"
server_key: "5ffb1b2c09f049fb821d231e11732bb5"
server_key_iv: "a43cdf426447c699f330dde5eef62478"
server_key_version: "encryptionkey@0"
I20260923 20:49:43.164794 19023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:43.190320 19023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:43.196098 19023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:43.196431 19023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:43.218325 19023 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.188:33645
I20260923 20:49:43.218374 19089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.188:33645 every 8 connection(s)
I20260923 20:49:43.220263 19023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
I20260923 20:49:43.223536 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19023
I20260923 20:49:43.223804 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790196359367181-5406-0/minicluster-data/master-2/wal/instance
I20260923 20:49:43.224419 5406 external_mini_cluster.cc:1428] Setting key 75d1310623da63d1a83709343b59019f
I20260923 20:49:43.225401 19090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:43.236733 19090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:43.252481 18941 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 4
I20260923 20:49:43.253211 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.226619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.5.71.188:33645)
0923 20:49:43.227463 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.227475 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.227514 (+ 39us) server_negotiation.cc:407] Connection header received
0923 20:49:43.229143 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.229168 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.229581 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.229945 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.232139 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.233924 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.236279 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.237217 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.237394 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.243463 (+ 6069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.243472 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.243484 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.243534 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.247735 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.248858 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.248865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.248874 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.249095 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.249602 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.249605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.249608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.250363 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.250695 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.251395 (+ 700us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.251898 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":404,"threads_started":1}
I20260923 20:49:43.260990 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.240405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.5.71.190:39533)
0923 20:49:43.241053 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.241060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.241574 (+ 514us) server_negotiation.cc:407] Connection header received
0923 20:49:43.241818 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.241823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.241902 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.242044 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.243751 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.244947 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.246711 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.247421 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.247597 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.251951 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.251957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.251960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.252006 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.257027 (+ 5021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.258033 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.258042 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.258046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.258251 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.258700 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.258709 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.258713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.259151 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.259324 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.259950 (+ 626us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.260302 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":311,"threads_started":1}
I20260923 20:49:43.261248 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.240381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:39533 (local address 127.0.0.1:58448)
0923 20:49:43.241427 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.241467 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:43.241675 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:43.242155 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:43.242164 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:43.243093 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:43.243539 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.243552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.245232 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.245238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.246537 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.246552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.247641 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.247648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.247879 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.248661 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:43.248689 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:43.251641 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:43.257236 (+ 5595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.257257 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.257265 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.257867 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.258388 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.258393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.258397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.258571 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.259302 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:43.259312 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:43.259526 (+ 214us) client_negotiation.cc:769] Sending connection context
0923 20:49:43.259773 (+ 247us) client_negotiation.cc:240] Negotiation successful
0923 20:49:43.260212 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":632,"threads_started":1}
I20260923 20:49:43.263769 19090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:43.272313 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.253137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54790)
0923 20:49:43.253296 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.253316 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:43.253433 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:43.254584 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:43.254592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:43.255020 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:43.255185 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.255194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.257465 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.257470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.258940 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.258953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.260013 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.260019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.260275 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.261258 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:43.261285 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:43.265131 (+ 3846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:43.269188 (+ 4057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.269194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.269199 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.269663 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.270114 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.270121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.270125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.270292 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.270917 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:43.270923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:43.271186 (+ 263us) client_negotiation.cc:769] Sending connection context
0923 20:49:43.271375 (+ 189us) client_negotiation.cc:240] Negotiation successful
0923 20:49:43.271778 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 20:49:43.272691 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.253421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.5.71.188:33645)
0923 20:49:43.254229 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.254235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.254258 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:49:43.254313 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.254321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.254402 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.254542 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.255372 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.257334 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.259077 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.259743 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.260838 (+ 1095us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.265421 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.265428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.265434 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.265474 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.269057 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.269859 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.269863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.269866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.269991 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.270401 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.270408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.270411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.270781 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.270920 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.272015 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.272257 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":430,"threads_started":1}
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:49:43.279019 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.256863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.5.71.190:39533)
0923 20:49:43.257579 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.257585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.257617 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:49:43.257668 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.257674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.257753 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.257863 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.259762 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.262031 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.264176 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.265131 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.265331 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.270835 (+ 5504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.270864 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.270869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.270916 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.275514 (+ 4598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.276604 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.276611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.276617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.276805 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.277239 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.277244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.277247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.277543 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.277878 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.278372 (+ 494us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.278611 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":402,"threads_started":1}
I20260923 20:49:43.280138 18941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: Bootstrap starting.
I20260923 20:49:43.283442 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.264642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51164 (local address 127.5.71.189:46791)
0923 20:49:43.265080 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.265087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.265231 (+ 144us) server_negotiation.cc:407] Connection header received
0923 20:49:43.265472 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.265478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.265569 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.265646 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.266871 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.268315 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.270534 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.271163 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.271333 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.275859 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.275871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.275876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.275923 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.279686 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.280554 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.280560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.280563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.280697 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.281076 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.281082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.281086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.281397 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.281556 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.282701 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.282974 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284}
I20260923 20:49:43.284521 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.264639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:46791 (local address 127.0.0.1:51164)
0923 20:49:43.265124 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.265147 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:43.265287 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:43.265896 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:43.265904 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:43.266460 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:43.266688 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.266710 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.268529 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.268537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.270342 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.270360 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.271344 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.271350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.271549 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.272263 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:43.272287 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:43.275578 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:43.279898 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.279906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.279909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.280433 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.280823 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.280828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.280832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.280965 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.281548 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:43.281553 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:43.282318 (+ 765us) client_negotiation.cc:769] Sending connection context
0923 20:49:43.283734 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0923 20:49:43.284143 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374}
I20260923 20:49:43.284934 18941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:43.285658 19090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:49:43.286491 18941 log.cc:824] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:43.290673 18941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: No bootstrap required, opened a new log
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196583, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:49:43.302001 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.282030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51174 (local address 127.5.71.189:46791)
0923 20:49:43.282845 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.282850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.282867 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:49:43.282907 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.282910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.282963 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.283074 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.284894 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.286107 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.287840 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.288479 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.288618 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.293816 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.293822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.293825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.293863 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.297404 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.298798 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.298807 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.298810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.298981 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.299481 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.299488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.299493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.299926 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.300230 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.301271 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.301528 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":525,"threads_started":1}
I20260923 20:49:43.302292 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.286547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.5.71.188:33645)
0923 20:49:43.286709 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.286717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.286792 (+ 75us) server_negotiation.cc:407] Connection header received
0923 20:49:43.287012 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.287017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.287118 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.287197 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.288268 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.289337 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.291056 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.291894 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.292070 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.296210 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.296218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.296222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.296266 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.299399 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.300253 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.300258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.300262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.300380 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.300740 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.300745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.300748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.301048 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.301191 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.301609 (+ 418us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.301962 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":45}
I20260923 20:49:43.302686 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.286460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54792)
0923 20:49:43.286707 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.286728 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:43.286879 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:43.287442 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:43.287447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:43.287955 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:43.288117 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.288127 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.289518 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.289524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.290886 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.290903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.292120 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.292129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.292332 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.293023 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:43.293051 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:43.295972 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:43.299583 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.299588 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.299592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.300130 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.300486 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.300492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.300495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.300621 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.301240 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:43.301244 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:43.301369 (+ 125us) client_negotiation.cc:769] Sending connection context
0923 20:49:43.301497 (+ 128us) client_negotiation.cc:240] Negotiation successful
0923 20:49:43.301823 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260923 20:49:43.302392 18941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.302838 18941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:43.302947 18941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3adaddfb83a24b7d8385e33d71808606, State: Initialized, Role: FOLLOWER
I20260923 20:49:43.303920 18941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.305763 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.306267 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:49:43.307287 18941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:43.308305 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:49:43.308753 19090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: Bootstrap starting.
I20260923 20:49:43.313493 19090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:43.316349 19090 log.cc:824] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:43.325238 19090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: No bootstrap required, opened a new log
W20260923 20:49:43.330833 19112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:49:43.331018 19112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:49:43.319749 18874 master_runner.cc:186] Error getting master registration for 127.5.71.189:46791: OK, instance_id { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" instance_seqno: 1790196582652351 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 20:49:43.332909 18874 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 20:49:43.343139 18895 tablet.cc:2369] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:43.343097 19090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.343683 19090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:43.343870 19090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45951424fb934e688e8c47df7339d663, State: Initialized, Role: FOLLOWER
I20260923 20:49:43.344908 19090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.352224 19120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.352766 19120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:49:43.355366 19090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:43.374099 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.338614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.5.71.189:46791)
0923 20:49:43.338874 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.338882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.351249 (+ 12367us) server_negotiation.cc:407] Connection header received
0923 20:49:43.352012 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.352020 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.352139 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.352232 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.355097 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.356384 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.359178 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.359938 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.360055 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.364684 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.364694 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.364699 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.364748 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.369401 (+ 4653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.370609 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.370615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.370619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.370811 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.371888 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.371896 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.371901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.372402 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.372677 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.373388 (+ 711us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.373669 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 20:49:43.384290 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.339014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.5.71.190:39533)
0923 20:49:43.339255 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.339262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.356417 (+ 17155us) server_negotiation.cc:407] Connection header received
0923 20:49:43.356776 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.356785 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.356878 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.356959 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.359883 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.361030 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.363674 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.364484 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.364606 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.373203 (+ 8597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.373214 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.373222 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.373280 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.379002 (+ 5722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.380256 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.380265 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.380271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.380483 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.381060 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.381068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.381074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.381878 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.382309 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.383224 (+ 915us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.383613 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:49:43.385946 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.356597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54796 (local address 127.5.71.188:33645)
0923 20:49:43.356772 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.356781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.364192 (+ 7411us) server_negotiation.cc:407] Connection header received
0923 20:49:43.364412 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.364421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.364563 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.364682 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.367191 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.368665 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.371833 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.372866 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.373007 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.377295 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.377305 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.377311 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.377365 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.381864 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.382854 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.382860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.382864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.383064 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.383526 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.383531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.383535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.383946 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.384250 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.385161 (+ 911us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.385507 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:49:43.392401 19015 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } attempt: 5
Sep 23 20:49:43 dist-test-slave-dn29 krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196582, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:49:43.418248 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.393168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:33645 (local address 127.0.0.1:54810)
0923 20:49:43.393397 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:43.393419 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:43.393563 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:43.394286 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:43.394297 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:43.395114 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:43.395435 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.395453 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.397735 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.397742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.400025 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:43.400050 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.401605 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.401612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:43.401862 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.404638 (+ 2776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:43.404670 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:43.408257 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:43.414657 (+ 6400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.414666 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.414672 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.415371 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.415964 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:43.415970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:43.415973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:43.416176 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:43.417022 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:43.417029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:43.417232 (+ 203us) client_negotiation.cc:769] Sending connection context
0923 20:49:43.417448 (+ 216us) client_negotiation.cc:240] Negotiation successful
0923 20:49:43.417799 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 20:49:43.419425 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:43.393278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.5.71.188:33645)
0923 20:49:43.393475 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:43.393482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:43.393505 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:49:43.393751 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:43.393761 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:43.393868 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:43.393998 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:43.395684 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.397504 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.400278 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:43.401177 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:43.401356 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:43.408526 (+ 7170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:43.408534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:43.408540 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:43.408594 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:43.414414 (+ 5820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.415553 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.415560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.415565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.415735 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:43.416323 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:43.416330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:43.416334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:43.416751 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:43.416936 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:43.418742 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0923 20:49:43.418996 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:49:43.406169 19023 master_runner.cc:186] Error getting master registration for 127.5.71.189:46791: OK, instance_id { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" instance_seqno: 1790196582652351 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 20:49:43.420878 19023 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 20:49:43.426376 19015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: Bootstrap starting.
W20260923 20:49:43.427053 19132 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:49:43.427205 19132 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:49:43.430262 19015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:43.431528 19015 log.cc:824] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:43.437130 19015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: No bootstrap required, opened a new log
I20260923 20:49:43.447250 19015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.447782 19015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:43.447945 19015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a446de820ee4ca588ed36cae5453b30, State: Initialized, Role: FOLLOWER
I20260923 20:49:43.448599 19100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:43.448716 19015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.448940 19100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.450582 19135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.450966 19100 leader_election.cc:288] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a446de820ee4ca588ed36cae5453b30 (127.5.71.189:46791), 45951424fb934e688e8c47df7339d663 (127.5.71.188:33645)
I20260923 20:49:43.451064 19135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 20:49:43.451438 19044 tablet.cc:2369] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:43.452472 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3adaddfb83a24b7d8385e33d71808606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45951424fb934e688e8c47df7339d663" is_pre_election: true
I20260923 20:49:43.452813 19015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:43.453100 19065 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3adaddfb83a24b7d8385e33d71808606 in term 0.
I20260923 20:49:43.452970 18990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3adaddfb83a24b7d8385e33d71808606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a446de820ee4ca588ed36cae5453b30" is_pre_election: true
I20260923 20:49:43.453569 18990 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3adaddfb83a24b7d8385e33d71808606 in term 0.
I20260923 20:49:43.453999 18891 leader_election.cc:302] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3adaddfb83a24b7d8385e33d71808606, 45951424fb934e688e8c47df7339d663; no voters:
I20260923 20:49:43.454525 19100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:49:43.454713 19100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:49:43.454820 19100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:43.458508 19100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.459856 18990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3adaddfb83a24b7d8385e33d71808606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a446de820ee4ca588ed36cae5453b30"
I20260923 20:49:43.460105 18990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:43.463644 18990 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3adaddfb83a24b7d8385e33d71808606 in term 1.
I20260923 20:49:43.464140 18892 leader_election.cc:302] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a446de820ee4ca588ed36cae5453b30, 3adaddfb83a24b7d8385e33d71808606; no voters:
I20260923 20:49:43.467012 19065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3adaddfb83a24b7d8385e33d71808606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45951424fb934e688e8c47df7339d663"
I20260923 20:49:43.467260 19065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:43.470853 19065 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3adaddfb83a24b7d8385e33d71808606 in term 1.
I20260923 20:49:43.471665 19100 leader_election.cc:288] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [CANDIDATE]: Term 1 election: Requested vote from peers 2a446de820ee4ca588ed36cae5453b30 (127.5.71.189:46791), 45951424fb934e688e8c47df7339d663 (127.5.71.188:33645)
I20260923 20:49:43.471956 19100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:43.472122 19100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [term 1 LEADER]: Becoming Leader. State: Replica: 3adaddfb83a24b7d8385e33d71808606, State: Running, Role: LEADER
I20260923 20:49:43.472791 19100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } }
I20260923 20:49:43.491602 19138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3adaddfb83a24b7d8385e33d71808606. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.492136 19138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:43.496416 19140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:43.498780 19140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:43.525128 19065 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [term 1 FOLLOWER]: Refusing update from remote peer 3adaddfb83a24b7d8385e33d71808606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:43.526268 19138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:43.530524 18990 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [term 1 FOLLOWER]: Refusing update from remote peer 3adaddfb83a24b7d8385e33d71808606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:43.531504 19138 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:43.543566 19120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3adaddfb83a24b7d8385e33d71808606. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.543956 19120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:49:43.548302 19142 mvcc.cc:205] Tried to move back new op lower bound from 7332645206083280896 to 7332645205919936512. Current Snapshot: MvccSnapshot[applied={T|T < 7332645206083280896}]
I20260923 20:49:43.550606 19145 mvcc.cc:205] Tried to move back new op lower bound from 7332645206083280896 to 7332645205919936512. Current Snapshot: MvccSnapshot[applied={T|T < 7332645206083280896}]
I20260923 20:49:43.552203 19135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3adaddfb83a24b7d8385e33d71808606. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.552307 19138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.552584 19135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:49:43.552636 19138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:43.559911 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.560328 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:43.560706 19140 catalog_manager.cc:1347] Generated new cluster ID: 3078a606cd5441e7a25ea1a06f67651e
I20260923 20:49:43.560814 19140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:49:43.570534 19120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.570890 19120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:49:43.573313 19135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3adaddfb83a24b7d8385e33d71808606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3adaddfb83a24b7d8385e33d71808606" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 39533 } } peers { permanent_uuid: "2a446de820ee4ca588ed36cae5453b30" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 46791 } } peers { permanent_uuid: "45951424fb934e688e8c47df7339d663" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 33645 } } }
I20260923 20:49:43.573625 19135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:49:43.623651 19140 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:49:43.625945 19140 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:49:43.639748 19157 catalog_manager.cc:1259] Loaded cluster ID: 3078a606cd5441e7a25ea1a06f67651e
I20260923 20:49:43.639976 19157 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: loading cluster ID for follower catalog manager: success
I20260923 20:49:43.644541 19157 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: acquiring CA information for follower catalog manager: success
W20260923 20:49:43.649698 19157 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260923 20:49:43.650204 19140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3adaddfb83a24b7d8385e33d71808606: Generated new TSK 0
I20260923 20:49:43.651518 19140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:49:44.429332 19132 catalog_manager.cc:1259] Loaded cluster ID: 3078a606cd5441e7a25ea1a06f67651e
I20260923 20:49:44.429437 19132 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: loading cluster ID for follower catalog manager: success
I20260923 20:49:44.431998 19132 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: acquiring CA information for follower catalog manager: success
I20260923 20:49:44.433849 19132 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45951424fb934e688e8c47df7339d663: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 20:49:44.652632 19157 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a446de820ee4ca588ed36cae5453b30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 20:49:44.783393 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18874
I20260923 20:49:44.785229 18936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:44.903843 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18874
I20260923 20:49:44.926314 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 18948
I20260923 20:49:44.927712 19010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:45.041632 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 18948
I20260923 20:49:45.063119 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19023
I20260923 20:49:45.064399 19085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:45.175330 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19023
2026-09-23T20:49:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4910 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:49:45 dist-test-slave-dn29 krb5kdc[19168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:49:45 dist-test-slave-dn29 krb5kdc[19168](info): set up 2 sockets
Sep 23 20:49:45 dist-test-slave-dn29 krb5kdc[19168](info): commencing operation
krb5kdc: starting...
W20260923 20:49:46.333096 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:49:46 dist-test-slave-dn29 krb5kdc[19168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:49:46.601696 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:46071
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:36757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:46071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:46.956544 19181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:46.956926 19181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:46.957105 19181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:46.965365 19181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:46.965535 19181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:46.965620 19181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:46.965720 19181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:46.965808 19181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:46.976110 19181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:36757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:46071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:46.977711 19181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:46.979105 19181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:46.993396 19186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:46.993388 19189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:46.993444 19187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:46.994479 19181 server_base.cc:1034] running on GCE node
I20260923 20:49:46.995383 19181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:46.997208 19181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:46.998493 19181 hybrid_clock.cc:648] HybridClock initialized: now 1790196586998443 us; error 40 us; skew 500 ppm
I20260923 20:49:46.998900 19181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:46 dist-test-slave-dn29 krb5kdc[19168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196586, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:47.002530 19181 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:47.004348 19181 webserver.cc:460] Webserver started at http://127.5.71.190:41299/ using document root <none> and password file <none>
I20260923 20:49:47.005246 19181 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:47.005405 19181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:47.005753 19181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:47.009251 19181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "8c5eeb9a178247cc9ad1cc19e6f29527"
format_stamp: "Formatted at 2026-09-23 20:49:46 on dist-test-slave-dn29"
server_key: "d4b4b6d105fef356587cc209c8a291da"
server_key_iv: "8337a5654ae23bb4740c5275499ce1ea"
server_key_version: "encryptionkey@0"
I20260923 20:49:47.010201 19181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "8c5eeb9a178247cc9ad1cc19e6f29527"
format_stamp: "Formatted at 2026-09-23 20:49:46 on dist-test-slave-dn29"
server_key: "d4b4b6d105fef356587cc209c8a291da"
server_key_iv: "8337a5654ae23bb4740c5275499ce1ea"
server_key_version: "encryptionkey@0"
I20260923 20:49:47.016921 19181 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260923 20:49:47.020900 19196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:47.022759 19181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260923 20:49:47.023000 19181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "8c5eeb9a178247cc9ad1cc19e6f29527"
format_stamp: "Formatted at 2026-09-23 20:49:46 on dist-test-slave-dn29"
server_key: "d4b4b6d105fef356587cc209c8a291da"
server_key_iv: "8337a5654ae23bb4740c5275499ce1ea"
server_key_version: "encryptionkey@0"
I20260923 20:49:47.023308 19181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:47.053383 19181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:47.058362 19181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:47.058683 19181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:47.079231 19181 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:46071
I20260923 20:49:47.079247 19247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:46071 every 8 connection(s)
I20260923 20:49:47.080874 19181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:47.085429 19248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:47.087016 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19181
I20260923 20:49:47.087265 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:49:47.088142 5406 external_mini_cluster.cc:1428] Setting key fe9e9cfb2fd4d97c7256e823e288bbf0
I20260923 20:49:47.100782 19248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527: Bootstrap starting.
Sep 23 20:49:47 dist-test-slave-dn29 krb5kdc[19168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:47.105479 19248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:47.107304 19248 log.cc:824] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:47.111267 19248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527: No bootstrap required, opened a new log
I20260923 20:49:47.115877 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:47.090357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.5.71.190:46071)
0923 20:49:47.091325 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:47.091341 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:47.091386 (+ 45us) server_negotiation.cc:407] Connection header received
0923 20:49:47.093560 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:47.093594 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:47.094061 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:47.094510 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:47.096198 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:47.097755 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:47.099188 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:47.099995 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:47.100188 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:47.106083 (+ 5895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:47.106092 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:47.106111 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:47.106156 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:47.110333 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:47.111205 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:47.111218 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:47.111231 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:47.111480 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:47.112073 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:47.112079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:47.112082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:47.112875 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:47.113240 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:47.113732 (+ 492us) server_negotiation.cc:299] Negotiation successful
0923 20:49:47.114276 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":412,"threads_started":1}
I20260923 20:49:47.120628 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:49:47.122455 19248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46071 } }
I20260923 20:49:47.122925 19248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:47.123066 19248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5eeb9a178247cc9ad1cc19e6f29527, State: Initialized, Role: FOLLOWER
I20260923 20:49:47.123771 19248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46071 } }
I20260923 20:49:47.124159 19248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:49:47.124329 19248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:49:47.124526 19248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:47.126978 19248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46071 } }
I20260923 20:49:47.127552 19248 leader_election.cc:302] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c5eeb9a178247cc9ad1cc19e6f29527; no voters:
I20260923 20:49:47.128315 19248 leader_election.cc:288] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:49:47.128472 19253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:47.129308 19253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [term 1 LEADER]: Becoming Leader. State: Replica: 8c5eeb9a178247cc9ad1cc19e6f29527, State: Running, Role: LEADER
I20260923 20:49:47.130043 19253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46071 } }
I20260923 20:49:47.130982 19248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:47.133281 19254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46071 } } }
I20260923 20:49:47.133255 19255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c5eeb9a178247cc9ad1cc19e6f29527. Latest consensus state: current_term: 1 leader_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5eeb9a178247cc9ad1cc19e6f29527" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46071 } } }
I20260923 20:49:47.133812 19254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:47.133836 19255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527 [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:47.135133 19258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:47.140846 19258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:47.152318 19258 catalog_manager.cc:1347] Generated new cluster ID: 3e77114e75934df6b2e2ad56909d7d39
I20260923 20:49:47.152446 19258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:49:47.187496 19258 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:49:47.189464 19258 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:49:47.201821 19258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c5eeb9a178247cc9ad1cc19e6f29527: Generated new TSK 0
I20260923 20:49:47.202956 19258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:49:47.234146 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19181
I20260923 20:49:47.235409 19243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:47.336354 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19181
2026-09-23T20:49:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2154 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:49:47 dist-test-slave-dn29 krb5kdc[19276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:49:47 dist-test-slave-dn29 krb5kdc[19276](info): set up 2 sockets
Sep 23 20:49:47 dist-test-slave-dn29 krb5kdc[19276](info): commencing operation
krb5kdc: starting...
W20260923 20:49:48.492887 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:49:48 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:49:48.748907 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45817
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:49.109794 19289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:49.110196 19289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:49.110370 19289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:49.118673 19289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:49.118860 19289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:49.118969 19289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:49.119071 19289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:49.119158 19289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:49.129994 19289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:49.131546 19289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:49.133035 19289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:49.146086 19294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:49.146209 19295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:49.146178 19297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:49.147500 19289 server_base.cc:1034] running on GCE node
I20260923 20:49:49.148499 19289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:49.150416 19289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:49.151712 19289 hybrid_clock.cc:648] HybridClock initialized: now 1790196589151584 us; error 102 us; skew 500 ppm
I20260923 20:49:49.152144 19289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:49 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196589, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:49.155958 19289 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:49.157738 19289 webserver.cc:460] Webserver started at http://127.5.71.190:38819/ using document root <none> and password file <none>
I20260923 20:49:49.158556 19289 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:49.158725 19289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:49.159108 19289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:49.162714 19289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "96f65377c9024acfbecc0c020ecc806c"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "bfc22df0f765267ad615db92d40f8f2e"
server_key_iv: "78ff449aa28fee0b80dfda7b0e836f79"
server_key_version: "encryptionkey@0"
I20260923 20:49:49.163595 19289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "96f65377c9024acfbecc0c020ecc806c"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "bfc22df0f765267ad615db92d40f8f2e"
server_key_iv: "78ff449aa28fee0b80dfda7b0e836f79"
server_key_version: "encryptionkey@0"
I20260923 20:49:49.169827 19289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:49:49.173978 19304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:49.175529 19289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:49:49.175846 19289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "96f65377c9024acfbecc0c020ecc806c"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "bfc22df0f765267ad615db92d40f8f2e"
server_key_iv: "78ff449aa28fee0b80dfda7b0e836f79"
server_key_version: "encryptionkey@0"
I20260923 20:49:49.176071 19289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:49.191972 19289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:49.196533 19289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:49.196854 19289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:49.217121 19289 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45817
I20260923 20:49:49.217119 19355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45817 every 8 connection(s)
I20260923 20:49:49.218676 19289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:49.222908 19356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:49.223639 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19289
I20260923 20:49:49.223889 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:49:49.224733 5406 external_mini_cluster.cc:1428] Setting key 95e807dadd4f0c50fc3ff1b8fe25a504
I20260923 20:49:49.237881 19356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap starting.
Sep 23 20:49:49 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:49.241797 19356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:49.243039 19356 log.cc:824] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:49.246773 19356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: No bootstrap required, opened a new log
I20260923 20:49:49.251899 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:49.227122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.5.71.190:45817)
0923 20:49:49.228271 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:49.228285 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:49.228332 (+ 47us) server_negotiation.cc:407] Connection header received
0923 20:49:49.230459 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:49.230487 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:49.231015 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:49.231490 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:49.233154 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:49.234620 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:49.236126 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:49.236902 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:49.237123 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:49.242373 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:49.242383 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:49.242401 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:49.242446 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:49.246400 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:49.247418 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:49.247426 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:49.247433 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:49.247668 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:49.248196 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:49.248201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:49.248203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:49.248954 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:49.249580 (+ 626us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:49.250055 (+ 475us) server_negotiation.cc:299] Negotiation successful
0923 20:49:49.250635 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":583,"threads_started":1}
I20260923 20:49:49.256455 19356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:49.256907 19356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:49.257030 19356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f65377c9024acfbecc0c020ecc806c, State: Initialized, Role: FOLLOWER
I20260923 20:49:49.257733 19356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:49.258000 19356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:49:49.258154 19356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:49:49.258345 19356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:49.261282 19356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:49.261860 19356 leader_election.cc:302] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96f65377c9024acfbecc0c020ecc806c; no voters:
I20260923 20:49:49.262512 19356 leader_election.cc:288] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:49:49.262645 19361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:49.264281 19361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 LEADER]: Becoming Leader. State: Replica: 96f65377c9024acfbecc0c020ecc806c, State: Running, Role: LEADER
I20260923 20:49:49.264984 19361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:49.265822 19356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:49.268792 19363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f65377c9024acfbecc0c020ecc806c. Latest consensus state: current_term: 1 leader_uuid: "96f65377c9024acfbecc0c020ecc806c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } } }
I20260923 20:49:49.268891 19362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96f65377c9024acfbecc0c020ecc806c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } } }
I20260923 20:49:49.269343 19363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:49.269387 19362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:49.271267 19368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:49.277890 19368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:49.288199 19368 catalog_manager.cc:1347] Generated new cluster ID: a37f6dea967d4382ae70cef140509556
I20260923 20:49:49.288335 19368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:49:49.301846 19368 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:49:49.303330 19368 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:49:49.314417 19368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Generated new TSK 0
I20260923 20:49:49.315414 19368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:49:49.388266 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45817
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:49.761237 19382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:49.761534 19382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:49.761629 19382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:49.769737 19382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:49.769923 19382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:49.770116 19382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:49:49.781654 19382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:49.783144 19382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:49.784729 19382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:49.799518 19388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:49.799621 19387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:49.799641 19390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:49.800392 19382 server_base.cc:1034] running on GCE node
I20260923 20:49:49.800980 19382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:49.801930 19382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:49.803162 19382 hybrid_clock.cc:648] HybridClock initialized: now 1790196589803097 us; error 55 us; skew 500 ppm
I20260923 20:49:49.803545 19382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:49 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196589, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:49.807014 19382 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:49:49.808816 19382 webserver.cc:460] Webserver started at http://127.5.71.129:46449/ using document root <none> and password file <none>
I20260923 20:49:49.809541 19382 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:49.809652 19382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:49.809954 19382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:49.813164 19382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "92e70ced720b74c1ba9024c8fa27c897"
server_key_iv: "823cfa8c9bbb4d9b8b3be3628b0c3f74"
server_key_version: "encryptionkey@0"
I20260923 20:49:49.814052 19382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "92e70ced720b74c1ba9024c8fa27c897"
server_key_iv: "823cfa8c9bbb4d9b8b3be3628b0c3f74"
server_key_version: "encryptionkey@0"
I20260923 20:49:49.820292 19382 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:49:49.824378 19397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:49.826040 19382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:49:49.826233 19382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "92e70ced720b74c1ba9024c8fa27c897"
server_key_iv: "823cfa8c9bbb4d9b8b3be3628b0c3f74"
server_key_version: "encryptionkey@0"
I20260923 20:49:49.826462 19382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:49.848971 19382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:49.854115 19382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:49.854466 19382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:49.855592 19382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:49.857264 19382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:49.857390 19382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:49.857569 19382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:49.857637 19382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:49.895131 19382 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:40927
I20260923 20:49:49.895229 19509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:40927 every 8 connection(s)
I20260923 20:49:49.896523 19382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:49:49.898891 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19382
I20260923 20:49:49.899108 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:49:49.899996 5406 external_mini_cluster.cc:1428] Setting key b8cd26c758215eeb90ba0ee2d00de2bd
Sep 23 20:49:49 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196589, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:49.920233 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:49.900370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:53331 (local address 127.5.71.190:45817)
0923 20:49:49.900954 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:49.900961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:49.901696 (+ 735us) server_negotiation.cc:407] Connection header received
0923 20:49:49.903777 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:49.903782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:49.903893 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:49.904013 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:49.905574 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:49.906304 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:49.908136 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:49.908543 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:49.908727 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:49.913846 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:49.913852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:49.913855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:49.913892 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:49.916757 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:49.917654 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:49.917659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:49.917662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:49.917807 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:49.918281 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:49.918286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:49.918291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:49.918573 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:49.918754 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:49.919540 (+ 786us) server_negotiation.cc:299] Negotiation successful
0923 20:49:49.919851 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":510,"threads_started":1}
I20260923 20:49:49.921375 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:49.900218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:53331)
0923 20:49:49.901517 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:49.901577 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:49.903495 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:49.904275 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:49.904284 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:49.904803 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:49.905398 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:49.905416 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:49.906479 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:49.906483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:49.907974 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:49.907987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:49.909043 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:49.909072 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:49.909388 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:49.910255 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:49.910297 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:49.913546 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:49.916970 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:49.916980 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:49.916995 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:49.917504 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:49.917932 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:49.917936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:49.917940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:49.918149 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:49.918724 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:49.918732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:49.919127 (+ 395us) client_negotiation.cc:769] Sending connection context
0923 20:49:49.919429 (+ 302us) client_negotiation.cc:240] Negotiation successful
0923 20:49:49.920042 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"thread_start_us":747,"threads_started":1}
I20260923 20:49:49.923156 19510 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45817
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:49:49.923529 19510 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:49.924355 19510 heartbeater.cc:511] Master 127.5.71.190:45817 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
I20260923 20:49:49.927075 19321 ts_manager.cc:194] Registered new tserver with Master: 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927)
I20260923 20:49:49.928680 19321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:53331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:49:49.956852 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45817
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:50.320065 19516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:50.320420 19516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:50.320578 19516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:50.328560 19516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:50.328718 19516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:50.328903 19516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:49:50.339387 19516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:50.340852 19516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:50.342216 19516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:50.356276 19522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:50.356236 19524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:50.356326 19521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:50.357152 19516 server_base.cc:1034] running on GCE node
I20260923 20:49:50.357832 19516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:50.358781 19516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:50.359966 19516 hybrid_clock.cc:648] HybridClock initialized: now 1790196590359898 us; error 58 us; skew 500 ppm
I20260923 20:49:50.360347 19516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:50 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:50.364256 19516 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:49:50.366011 19516 webserver.cc:460] Webserver started at http://127.5.71.130:46851/ using document root <none> and password file <none>
I20260923 20:49:50.366895 19516 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:50.367057 19516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:50.367424 19516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:50.370858 19516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "db2f2ff1dd7949128c285a8c98295ae3"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "567144c5020c61e47ea317f4eaac4457"
server_key_iv: "0a2ce4f45737bee36a3a5ef3185cc514"
server_key_version: "encryptionkey@0"
I20260923 20:49:50.371847 19516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "db2f2ff1dd7949128c285a8c98295ae3"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "567144c5020c61e47ea317f4eaac4457"
server_key_iv: "0a2ce4f45737bee36a3a5ef3185cc514"
server_key_version: "encryptionkey@0"
I20260923 20:49:50.378053 19516 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260923 20:49:50.382366 19531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:50.384075 19516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:49:50.384268 19516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "db2f2ff1dd7949128c285a8c98295ae3"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "567144c5020c61e47ea317f4eaac4457"
server_key_iv: "0a2ce4f45737bee36a3a5ef3185cc514"
server_key_version: "encryptionkey@0"
I20260923 20:49:50.384497 19516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:50.399293 19516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:50.404263 19516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:50.404587 19516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:50.405690 19516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:50.407203 19516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:50.407315 19516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:50.407469 19516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:50.407547 19516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:50.444303 19516 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:40607
I20260923 20:49:50.444398 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:40607 every 8 connection(s)
I20260923 20:49:50.445654 19516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:49:50.453306 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19516
I20260923 20:49:50.453543 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:49:50.454229 5406 external_mini_cluster.cc:1428] Setting key 7c5b6eef28264bce54893ddec0866e7d
Sep 23 20:49:50 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:50.471714 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:50.448543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:39721 (local address 127.5.71.190:45817)
0923 20:49:50.449017 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:50.449023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:50.450165 (+ 1142us) server_negotiation.cc:407] Connection header received
0923 20:49:50.452275 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:50.452280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:50.452381 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:50.452489 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:50.455181 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:50.456568 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:50.459406 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:50.460173 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:50.460294 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:50.465258 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:50.465264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:50.465267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:50.465306 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:50.468298 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:50.469135 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:50.469148 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:50.469151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:50.469300 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:50.469781 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:50.469787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:50.469790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:50.470077 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:50.470243 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:50.471025 (+ 782us) server_negotiation.cc:299] Negotiation successful
0923 20:49:50.471307 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":227,"threads_started":1}
I20260923 20:49:50.473033 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:50.449247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:39721)
0923 20:49:50.449997 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:50.450036 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:50.451922 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:50.452940 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:50.452958 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:50.453898 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:50.454906 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:50.454946 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:50.456810 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:50.456816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:50.459175 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:50.459197 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:50.460441 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:50.460451 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:50.460856 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:50.461742 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:50.461783 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:50.464935 (+ 3152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:50.468482 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:50.468493 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:50.468513 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:50.468988 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:50.469427 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:50.469432 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:50.469436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:50.469653 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:50.470226 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:50.470235 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:50.470638 (+ 403us) client_negotiation.cc:769] Sending connection context
0923 20:49:50.470942 (+ 304us) client_negotiation.cc:240] Negotiation successful
0923 20:49:50.471552 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":288,"threads_started":1}
I20260923 20:49:50.474900 19644 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45817
I20260923 20:49:50.475263 19644 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:50.476209 19644 heartbeater.cc:511] Master 127.5.71.190:45817 requested a full tablet report, sending...
I20260923 20:49:50.477967 19321 ts_manager.cc:194] Registered new tserver with Master: db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
I20260923 20:49:50.478814 19321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:39721
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:49:50.516485 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45817
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:50.892225 19650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:50.892532 19650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:50.892693 19650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:50.900548 19650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:50.900672 19650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:50.900882 19650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:49:50.911849 19650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:50.913394 19650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:50.915450 19650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 20:49:50.932539 19510 heartbeater.cc:503] Master 127.5.71.190:45817 was elected leader, sending a full tablet report...
W20260923 20:49:50.933514 19655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:50.934360 19656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:50.939515 19658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:50.941697 19650 server_base.cc:1034] running on GCE node
I20260923 20:49:50.942438 19650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:50.943630 19650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:50.944902 19650 hybrid_clock.cc:648] HybridClock initialized: now 1790196590944850 us; error 55 us; skew 500 ppm
I20260923 20:49:50.945266 19650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:50 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:50.948748 19650 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:49:50.950495 19650 webserver.cc:460] Webserver started at http://127.5.71.131:38065/ using document root <none> and password file <none>
I20260923 20:49:50.951279 19650 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:50.951432 19650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:50.951757 19650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:49:50.954952 19650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "f1c2072da61e4ae9a0c73a1811d17023"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "6957487e687919f1ef7b2edc943c9b45"
server_key_iv: "f453b34f52ec06a950d610dae8f458f6"
server_key_version: "encryptionkey@0"
I20260923 20:49:50.955857 19650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "f1c2072da61e4ae9a0c73a1811d17023"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "6957487e687919f1ef7b2edc943c9b45"
server_key_iv: "f453b34f52ec06a950d610dae8f458f6"
server_key_version: "encryptionkey@0"
I20260923 20:49:50.962158 19650 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:49:50.966213 19665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:50.967978 19650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:49:50.968161 19650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "f1c2072da61e4ae9a0c73a1811d17023"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "6957487e687919f1ef7b2edc943c9b45"
server_key_iv: "f453b34f52ec06a950d610dae8f458f6"
server_key_version: "encryptionkey@0"
I20260923 20:49:50.968386 19650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:50.990757 19650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:50.995692 19650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:50.996112 19650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:50.997287 19650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:50.998867 19650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:50.998944 19650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:50.999087 19650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:50.999147 19650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:51.037858 19650 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:41749
I20260923 20:49:51.037906 19777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:41749 every 8 connection(s)
I20260923 20:49:51.039278 19650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:49:51.048533 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19650
I20260923 20:49:51.048750 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:49:51.049228 5406 external_mini_cluster.cc:1428] Setting key 437d6254425333dbc55104f6be16b16f
Sep 23 20:49:51 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196590, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:51.063827 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.042133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:34263 (local address 127.5.71.190:45817)
0923 20:49:51.042618 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.042623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.043992 (+ 1369us) server_negotiation.cc:407] Connection header received
0923 20:49:51.046166 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.046173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.046274 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.046409 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:51.048316 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.049344 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.051529 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.052092 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.052237 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.056660 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:51.056665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:51.056674 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:51.056713 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:51.059767 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.060964 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.060970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.060974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.061132 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.061764 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.061767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.061770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.062103 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:51.062267 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.063051 (+ 784us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.063260 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":249,"threads_started":1}
I20260923 20:49:51.064899 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.042662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:34263)
0923 20:49:51.043701 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:51.043747 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:51.045815 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:51.046703 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:51.046713 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:51.047341 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:51.048102 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.048125 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.049520 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.049525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.051314 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.051333 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.052370 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.052378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.052683 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.053552 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:51.053583 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:51.056238 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:51.060034 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:51.060042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:51.060059 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:51.060580 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:51.061288 (+ 708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:51.061293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:51.061295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:51.061535 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:51.062275 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:51.062282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:51.062662 (+ 380us) client_negotiation.cc:769] Sending connection context
0923 20:49:51.062920 (+ 258us) client_negotiation.cc:240] Negotiation successful
0923 20:49:51.063424 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":442,"threads_started":1}
I20260923 20:49:51.066671 19778 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45817
I20260923 20:49:51.067054 19778 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:51.068005 19778 heartbeater.cc:511] Master 127.5.71.190:45817 requested a full tablet report, sending...
I20260923 20:49:51.069852 19321 ts_manager.cc:194] Registered new tserver with Master: f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749)
I20260923 20:49:51.070982 19321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:34263
I20260923 20:49:51.079545 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:49:51.100266 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.084549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.5.71.190:45817)
0923 20:49:51.084813 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.084821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.085136 (+ 315us) server_negotiation.cc:407] Connection header received
0923 20:49:51.085318 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.085322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.085392 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.085480 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:51.087049 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.088160 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.090039 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.090495 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.090630 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.093586 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:51.093592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:51.093596 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:51.093639 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:51.096763 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.097732 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.097736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.097739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.097888 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.098405 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.098410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.098412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.098831 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:51.099055 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.099500 (+ 445us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.099905 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 20:49:51.104858 19321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:49:51.112056 19321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:49:51.152194 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.126756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41749 (local address 127.0.0.1:45650)
0923 20:49:51.141176 (+ 14420us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:51.141204 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:51.142342 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:51.143630 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:51.143639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:51.143685 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:51.145076 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.145088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.146863 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.146869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.149173 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.149193 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.151372 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.151377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.151554 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.151589 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 20:49:51.151660 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 20:49:51.151771 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14228,"mutex_wait_us":160,"thread_start_us":14922,"threads_started":1}
I20260923 20:49:51.152455 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.126388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.5.71.130:40607)
0923 20:49:51.143274 (+ 16886us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.143284 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.143316 (+ 32us) server_negotiation.cc:407] Connection header received
0923 20:49:51.143417 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.143424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.143661 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.144083 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:51.144990 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.146619 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.149385 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.150847 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.151007 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.151214 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.151944 (+ 730us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.152120 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16739,"thread_start_us":16619,"threads_started":1}
I20260923 20:49:51.152783 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.127560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45650 (local address 127.5.71.131:41749)
0923 20:49:51.136930 (+ 9370us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.136946 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.141361 (+ 4415us) server_negotiation.cc:407] Connection header received
0923 20:49:51.142602 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.142629 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.142966 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.143175 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:51.145274 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.146741 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.149365 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.150695 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.150844 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.151504 (+ 660us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.151777 (+ 273us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.151989 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8104,"thread_start_us":7901,"threads_started":1}
I20260923 20:49:51.153003 19793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.126282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40607 (local address 127.0.0.1:59630)
0923 20:49:51.141029 (+ 14747us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:51.141099 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:51.141413 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:51.143933 (+ 2520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:51.143939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:51.143976 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:51.144789 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.144807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.146777 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.146785 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.149167 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.149185 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.151046 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.151053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.151265 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.151355 (+ 90us) client_negotiation.cc:769] Sending connection context
0923 20:49:51.151495 (+ 140us) client_negotiation.cc:240] Negotiation successful
0923 20:49:51.151725 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14301,"mutex_wait_us":112,"thread_start_us":15124,"threads_started":1}
I20260923 20:49:51.154973 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.128714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.5.71.129:40927)
0923 20:49:51.143737 (+ 15023us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.143746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.144835 (+ 1089us) server_negotiation.cc:407] Connection header received
0923 20:49:51.145066 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.145077 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.145317 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.145518 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:51.146352 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.148026 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.150749 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.152360 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.152477 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.152702 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.154443 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.154629 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14830,"thread_start_us":14678,"threads_started":1}
I20260923 20:49:51.155665 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.127940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:42728)
0923 20:49:51.144723 (+ 16783us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:51.144743 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:51.144873 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:51.145753 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:51.145758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:51.145782 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:51.146159 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.146171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.148174 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.148180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.150537 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.150552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.152856 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.152865 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.153071 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.153108 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 20:49:51.153609 (+ 501us) client_negotiation.cc:240] Negotiation successful
0923 20:49:51.155313 (+ 1704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16601,"mutex_wait_us":58,"thread_start_us":16060,"threads_started":1}
I20260923 20:49:51.157023 19579 tablet_service.cc:1467] Processing CreateTablet for tablet faf788784f614d698e740be7d67b44f6 (DEFAULT_TABLE table=test-table [id=b10336dfae254d81a5d08e3a1739032e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:51.157037 19713 tablet_service.cc:1467] Processing CreateTablet for tablet faf788784f614d698e740be7d67b44f6 (DEFAULT_TABLE table=test-table [id=b10336dfae254d81a5d08e3a1739032e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:51.158273 19579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf788784f614d698e740be7d67b44f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:51.158454 19713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf788784f614d698e740be7d67b44f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:51.159531 19445 tablet_service.cc:1467] Processing CreateTablet for tablet faf788784f614d698e740be7d67b44f6 (DEFAULT_TABLE table=test-table [id=b10336dfae254d81a5d08e3a1739032e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:51.160967 19445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf788784f614d698e740be7d67b44f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:51.170047 19797 tablet_bootstrap.cc:492] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Bootstrap starting.
I20260923 20:49:51.174861 19797 tablet_bootstrap.cc:654] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:51.176090 19799 tablet_bootstrap.cc:492] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Bootstrap starting.
I20260923 20:49:51.176218 19797 log.cc:824] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:51.178068 19798 tablet_bootstrap.cc:492] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Bootstrap starting.
I20260923 20:49:51.181167 19799 tablet_bootstrap.cc:654] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:51.181516 19798 tablet_bootstrap.cc:654] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:51.182909 19798 log.cc:824] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:51.182971 19799 log.cc:824] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:51.186359 19798 tablet_bootstrap.cc:492] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: No bootstrap required, opened a new log
I20260923 20:49:51.186422 19797 tablet_bootstrap.cc:492] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: No bootstrap required, opened a new log
I20260923 20:49:51.186420 19799 tablet_bootstrap.cc:492] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: No bootstrap required, opened a new log
I20260923 20:49:51.186687 19798 ts_tablet_manager.cc:1397] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260923 20:49:51.186694 19797 ts_tablet_manager.cc:1397] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260923 20:49:51.186694 19799 ts_tablet_manager.cc:1397] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260923 20:49:51.194380 19798 raft_consensus.cc:348] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.194831 19798 raft_consensus.cc:374] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:51.194932 19798 raft_consensus.cc:729] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1c2072da61e4ae9a0c73a1811d17023, State: Initialized, Role: FOLLOWER
I20260923 20:49:51.195571 19798 consensus_queue.cc:260] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.196758 19778 heartbeater.cc:503] Master 127.5.71.190:45817 was elected leader, sending a full tablet report...
I20260923 20:49:51.197497 19798 ts_tablet_manager.cc:1428] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260923 20:49:51.197700 19799 raft_consensus.cc:348] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.198175 19799 raft_consensus.cc:374] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:51.198269 19799 raft_consensus.cc:729] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef2a69fc4bb442b8f7fe429b3def5c4, State: Initialized, Role: FOLLOWER
I20260923 20:49:51.199008 19799 consensus_queue.cc:260] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.199958 19797 raft_consensus.cc:348] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.200423 19797 raft_consensus.cc:374] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:51.200519 19797 raft_consensus.cc:729] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2f2ff1dd7949128c285a8c98295ae3, State: Initialized, Role: FOLLOWER
I20260923 20:49:51.200927 19799 ts_tablet_manager.cc:1428] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260923 20:49:51.201102 19797 consensus_queue.cc:260] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.202242 19644 heartbeater.cc:503] Master 127.5.71.190:45817 was elected leader, sending a full tablet report...
I20260923 20:49:51.202878 19797 ts_tablet_manager.cc:1428] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260923 20:49:51.291184 19779 tablet.cc:2369] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:51.385744 19804 raft_consensus.cc:482] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:51.386057 19804 raft_consensus.cc:504] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.387984 19804 leader_election.cc:288] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607), f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749)
I20260923 20:49:51.395843 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.388576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:42777 (local address 127.5.71.131:41749)
0923 20:49:51.388915 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.388924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.389551 (+ 627us) server_negotiation.cc:407] Connection header received
0923 20:49:51.389777 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.389783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.389864 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.389971 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:51.390811 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.392002 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.393992 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.394795 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.394982 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.395072 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.395405 (+ 333us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.395541 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260923 20:49:51.395843 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.388588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:44437 (local address 127.5.71.130:40607)
0923 20:49:51.388835 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.388843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.389491 (+ 648us) server_negotiation.cc:407] Connection header received
0923 20:49:51.389819 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.389825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.389901 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.389989 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:51.390811 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.392060 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.393974 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.394782 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.394974 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.395059 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.395377 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.395507 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:49:51.396070 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.388258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40607 (local address 127.5.71.129:44437)
0923 20:49:51.389357 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:51.389379 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:51.389563 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:51.390020 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:51.390026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:51.390056 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:51.390644 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.390652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.392250 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.392254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.393800 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.393816 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.395028 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.395034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.395208 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.395234 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 20:49:51.395348 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 20:49:51.395494 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":27,"thread_start_us":874,"threads_started":1}
I20260923 20:49:51.396097 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.388429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41749 (local address 127.5.71.129:42777)
0923 20:49:51.389442 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:51.389458 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:51.389624 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:51.390020 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:51.390026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:51.390067 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:51.390637 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.390650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.392145 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.392151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.393815 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:51.393831 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.395069 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.395075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:51.395238 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.395261 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 20:49:51.395375 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 20:49:51.395511 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":70,"spinlock_wait_cycles":60032,"thread_start_us":494,"threads_started":1}
I20260923 20:49:51.397181 19599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf788784f614d698e740be7d67b44f6" candidate_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2f2ff1dd7949128c285a8c98295ae3" is_pre_election: true
I20260923 20:49:51.397289 19733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf788784f614d698e740be7d67b44f6" candidate_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1c2072da61e4ae9a0c73a1811d17023" is_pre_election: true
I20260923 20:49:51.397722 19599 raft_consensus.cc:2454] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ef2a69fc4bb442b8f7fe429b3def5c4 in term 0.
I20260923 20:49:51.397789 19733 raft_consensus.cc:2454] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ef2a69fc4bb442b8f7fe429b3def5c4 in term 0.
I20260923 20:49:51.398497 19399 leader_election.cc:302] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4, db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023; no voters:
I20260923 20:49:51.398950 19804 raft_consensus.cc:2781] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:49:51.399117 19804 raft_consensus.cc:482] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:49:51.399250 19804 raft_consensus.cc:3037] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Advancing to term 1
W20260923 20:49:51.399312 19511 tablet.cc:2369] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:51.401777 19804 raft_consensus.cc:504] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.402415 19804 leader_election.cc:288] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 1 election: Requested vote from peers db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607), f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749)
I20260923 20:49:51.402943 19599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf788784f614d698e740be7d67b44f6" candidate_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2f2ff1dd7949128c285a8c98295ae3"
I20260923 20:49:51.402987 19733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faf788784f614d698e740be7d67b44f6" candidate_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1c2072da61e4ae9a0c73a1811d17023"
I20260923 20:49:51.403196 19733 raft_consensus.cc:3037] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:51.403195 19599 raft_consensus.cc:3037] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:51.406697 19599 raft_consensus.cc:2454] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ef2a69fc4bb442b8f7fe429b3def5c4 in term 1.
I20260923 20:49:51.406701 19733 raft_consensus.cc:2454] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ef2a69fc4bb442b8f7fe429b3def5c4 in term 1.
I20260923 20:49:51.407341 19400 leader_election.cc:302] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4, db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023; no voters:
I20260923 20:49:51.407703 19804 raft_consensus.cc:2781] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:51.408209 19804 raft_consensus.cc:686] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 LEADER]: Becoming Leader. State: Replica: 6ef2a69fc4bb442b8f7fe429b3def5c4, State: Running, Role: LEADER
I20260923 20:49:51.408812 19804 consensus_queue.cc:237] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } }
I20260923 20:49:51.413410 19320 catalog_manager.cc:5495] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } health_report { overall_health: UNKNOWN } } }
W20260923 20:49:51.448092 19645 tablet.cc:2369] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:51.453711 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.447069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.5.71.129:40927)
0923 20:49:51.447317 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.447325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.447347 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:49:51.447456 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.447461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.447570 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.447754 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:49:51.448312 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.449409 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.450968 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.451671 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.451827 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.452413 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:49:51.453095 (+ 682us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:49:51.453271 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.453330 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.453432 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260923 20:49:51.463330 19733 raft_consensus.cc:1261] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Refusing update from remote peer 6ef2a69fc4bb442b8f7fe429b3def5c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:51.463328 19599 raft_consensus.cc:1261] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Refusing update from remote peer 6ef2a69fc4bb442b8f7fe429b3def5c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:51.464747 19804 consensus_queue.cc:1035] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:51.465144 19808 consensus_queue.cc:1035] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:51.477664 19811 mvcc.cc:205] Tried to move back new op lower bound from 7332645238621089792 to 7332645238414622720. Current Snapshot: MvccSnapshot[applied={T|T < 7332645238621089792}]
I20260923 20:49:51.478317 19812 mvcc.cc:205] Tried to move back new op lower bound from 7332645238621089792 to 7332645238414622720. Current Snapshot: MvccSnapshot[applied={T|T < 7332645238621089792}]
I20260923 20:49:51.484172 19814 mvcc.cc:205] Tried to move back new op lower bound from 7332645238621089792 to 7332645238414622720. Current Snapshot: MvccSnapshot[applied={T|T < 7332645238621089792}]
I20260923 20:49:51.497015 19320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:49:51.497365 19320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:49:51.501045 19320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Sending DeleteTablet for 3 replicas of tablet faf788784f614d698e740be7d67b44f6
I20260923 20:49:51.502354 19579 tablet_service.cc:1514] Processing DeleteTablet for tablet faf788784f614d698e740be7d67b44f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:51 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:59630
I20260923 20:49:51.502360 19445 tablet_service.cc:1514] Processing DeleteTablet for tablet faf788784f614d698e740be7d67b44f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:51 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:42728
I20260923 20:49:51.502769 19713 tablet_service.cc:1514] Processing DeleteTablet for tablet faf788784f614d698e740be7d67b44f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:51 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:45650
I20260923 20:49:51.503993 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19382
I20260923 20:49:51.515754 19821 tablet_replica.cc:330] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: stopping tablet replica
I20260923 20:49:51.516546 19821 raft_consensus.cc:2229] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:49:51.517755 19821 raft_consensus.cc:2258] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:51.518810 19822 tablet_replica.cc:330] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: stopping tablet replica
I20260923 20:49:51.519218 19822 raft_consensus.cc:2229] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:49:51.519737 19822 raft_consensus.cc:2258] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:51.521768 19821 ts_tablet_manager.cc:1905] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:51.522038 19822 ts_tablet_manager.cc:1905] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:51.529443 19821 ts_tablet_manager.cc:1918] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:51.529423 19822 ts_tablet_manager.cc:1918] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:51.529645 19821 log.cc:1196] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/faf788784f614d698e740be7d67b44f6
I20260923 20:49:51.529832 19822 log.cc:1196] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/faf788784f614d698e740be7d67b44f6
I20260923 20:49:51.530246 19821 ts_tablet_manager.cc:1939] T faf788784f614d698e740be7d67b44f6 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Deleting consensus metadata
I20260923 20:49:51.530586 19822 ts_tablet_manager.cc:1939] T faf788784f614d698e740be7d67b44f6 P db2f2ff1dd7949128c285a8c98295ae3: Deleting consensus metadata
I20260923 20:49:51.532167 19823 tablet_replica.cc:330] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: stopping tablet replica
I20260923 20:49:51.532696 19305 catalog_manager.cc:4841] TS 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927): tablet faf788784f614d698e740be7d67b44f6 (table test-table [id=b10336dfae254d81a5d08e3a1739032e]) successfully deleted
I20260923 20:49:51.532850 19823 raft_consensus.cc:2229] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:49:51.533233 19306 catalog_manager.cc:4841] TS db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): tablet faf788784f614d698e740be7d67b44f6 (table test-table [id=b10336dfae254d81a5d08e3a1739032e]) successfully deleted
I20260923 20:49:51.533396 19823 raft_consensus.cc:2258] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:51.535696 19823 ts_tablet_manager.cc:1905] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 20:49:51 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:49:51.543265 19823 ts_tablet_manager.cc:1918] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:51.543459 19823 log.cc:1196] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/faf788784f614d698e740be7d67b44f6
I20260923 20:49:51.544056 19823 ts_tablet_manager.cc:1939] T faf788784f614d698e740be7d67b44f6 P f1c2072da61e4ae9a0c73a1811d17023: Deleting consensus metadata
I20260923 20:49:51.546111 19307 catalog_manager.cc:4841] TS f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): tablet faf788784f614d698e740be7d67b44f6 (table test-table [id=b10336dfae254d81a5d08e3a1739032e]) successfully deleted
I20260923 20:49:51.549278 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.505121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.5.71.129:40927)
0923 20:49:51.505305 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.505312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.530291 (+ 24979us) server_negotiation.cc:407] Connection header received
0923 20:49:51.530926 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.530933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.531017 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.531089 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:51.534737 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.535543 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.537103 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.537579 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.537730 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.541974 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:51.541990 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:51.542008 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:51.542054 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:51.545867 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.546733 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.546744 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.546748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.546941 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.547388 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.547400 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.547403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.547969 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:51.548281 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.548628 (+ 347us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.548875 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:49:51.550134 19505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:51.656049 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19382
I20260923 20:49:51.678521 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19516
Sep 23 20:49:51 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:49:51.702943 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.679525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.5.71.130:40607)
0923 20:49:51.679742 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.679747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.679916 (+ 169us) server_negotiation.cc:407] Connection header received
0923 20:49:51.680228 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.680234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.680342 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.680495 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:51.682344 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.683313 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.685286 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.685744 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.685923 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.695912 (+ 9989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:51.695931 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:51.695953 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:51.695999 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:51.699653 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.700651 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.700659 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.700661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.700800 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.701214 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.701219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.701221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.701621 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:51.701957 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.702253 (+ 296us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.702479 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:49:51.703958 19639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:51.798418 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19516
I20260923 20:49:51.816100 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19650
Sep 23 20:49:51 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:49:51.833310 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:51.816929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.5.71.131:41749)
0923 20:49:51.817160 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:51.817165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:51.817256 (+ 91us) server_negotiation.cc:407] Connection header received
0923 20:49:51.817440 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:51.817445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:51.817527 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:51.817620 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:51.818914 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.819666 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.820915 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:51.821316 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:51.821411 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:51.826553 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:51.826568 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:51.826588 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:51.826627 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:51.829892 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.830736 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.830747 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.830750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.830939 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:51.831400 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:51.831404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:51.831406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:51.832024 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:51.832308 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:51.832720 (+ 412us) server_negotiation.cc:299] Negotiation successful
0923 20:49:51.832973 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260923 20:49:51.834156 19773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:51.928400 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19650
I20260923 20:49:51.945947 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19289
I20260923 20:49:51.947402 19351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:52.071719 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19289
I20260923 20:49:52.096939 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45817
--webserver_interface=127.5.71.190
--webserver_port=38819
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:52.464747 19829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:52.465118 19829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:52.465297 19829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:52.472926 19829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:52.473058 19829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:52.473150 19829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:52.473239 19829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:52.473320 19829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:52.484562 19829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=38819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:52.486025 19829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:52.487496 19829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:52.517974 19835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:52.517854 19836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:52.518959 19838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:52.520790 19829 server_base.cc:1034] running on GCE node
I20260923 20:49:52.521838 19829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:52.523860 19829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:52.525184 19829 hybrid_clock.cc:648] HybridClock initialized: now 1790196592525115 us; error 74 us; skew 500 ppm
I20260923 20:49:52.525679 19829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:52 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196592, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:52.535480 19829 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:52.537883 19829 webserver.cc:460] Webserver started at http://127.5.71.190:38819/ using document root <none> and password file <none>
I20260923 20:49:52.538848 19829 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:52.539054 19829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:52.546643 19829 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260923 20:49:52.550583 19845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:52.552757 19829 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260923 20:49:52.552987 19829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "96f65377c9024acfbecc0c020ecc806c"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "bfc22df0f765267ad615db92d40f8f2e"
server_key_iv: "78ff449aa28fee0b80dfda7b0e836f79"
server_key_version: "encryptionkey@0"
I20260923 20:49:52.554309 19829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:52.596802 19829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:52.602174 19829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:52.602514 19829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:52.624691 19829 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45817
I20260923 20:49:52.624697 19896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45817 every 8 connection(s)
I20260923 20:49:52.626412 19829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:52.634187 19897 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:49:52.635135 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19829
I20260923 20:49:52.636010 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:40927
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:45817
--webserver_port=46449
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:52.638059 19897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap starting.
I20260923 20:49:52.666635 19897 log.cc:824] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:52.683748 19897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:49:52.684409 19897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap complete.
I20260923 20:49:52.694113 19897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:52.695055 19897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f65377c9024acfbecc0c020ecc806c, State: Initialized, Role: FOLLOWER
I20260923 20:49:52.695780 19897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:52.696148 19897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:49:52.696300 19897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:49:52.696487 19897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 1 FOLLOWER]: Advancing to term 2
I20260923 20:49:52.700641 19897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:52.701251 19897 leader_election.cc:302] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96f65377c9024acfbecc0c020ecc806c; no voters:
I20260923 20:49:52.701987 19897 leader_election.cc:288] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 20:49:52.702128 19901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Leader election won for term 2
I20260923 20:49:52.703403 19901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 LEADER]: Becoming Leader. State: Replica: 96f65377c9024acfbecc0c020ecc806c, State: Running, Role: LEADER
I20260923 20:49:52.704133 19901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:52.704639 19897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:52.706748 19903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f65377c9024acfbecc0c020ecc806c. Latest consensus state: current_term: 2 leader_uuid: "96f65377c9024acfbecc0c020ecc806c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } } }
I20260923 20:49:52.707144 19903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:52.707715 19902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "96f65377c9024acfbecc0c020ecc806c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } } }
I20260923 20:49:52.708037 19902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:52.721017 19907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:52.731562 19907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:52.733765 19907 catalog_manager.cc:1259] Loaded cluster ID: a37f6dea967d4382ae70cef140509556
I20260923 20:49:52.734026 19907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:49:52.739012 19907 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 20:49:52.760239 19907 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 20:49:52.770716 19907 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:49:52.772861 19907 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Loaded TSK: 0
I20260923 20:49:52.774785 19907 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 20:49:53.036824 19899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:53.037202 19899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:53.037380 19899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:53.045585 19899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:53.045760 19899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:53.045980 19899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:49:53.056983 19899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:40927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=46449
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:53.058531 19899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:53.060091 19899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:53.076483 19924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:53.076493 19925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:53.077932 19899 server_base.cc:1034] running on GCE node
W20260923 20:49:53.078465 19927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:53.079509 19899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:53.081136 19899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:53.082420 19899 hybrid_clock.cc:648] HybridClock initialized: now 1790196593082313 us; error 104 us; skew 500 ppm
I20260923 20:49:53.082871 19899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:53 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196593, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:53.087690 19899 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:49:53.089946 19899 webserver.cc:460] Webserver started at http://127.5.71.129:46449/ using document root <none> and password file <none>
I20260923 20:49:53.090951 19899 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:53.091116 19899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:53.099372 19899 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:49:53.103506 19934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:53.105736 19899 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260923 20:49:53.105980 19899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "92e70ced720b74c1ba9024c8fa27c897"
server_key_iv: "823cfa8c9bbb4d9b8b3be3628b0c3f74"
server_key_version: "encryptionkey@0"
I20260923 20:49:53.108007 19899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:53.139739 19899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:53.145762 19899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:53.146068 19899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:53.147373 19899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:53.149320 19899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:53.149471 19899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:53.149583 19899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:53.149672 19899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:53.202937 19899 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:40927
I20260923 20:49:53.202975 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:40927 every 8 connection(s)
I20260923 20:49:53.205343 19899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:49:53.212508 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 19899
I20260923 20:49:53.213320 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:40607
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:45817
--webserver_port=46851
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:49:53 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196593, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:53.259670 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:53.216787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40771 (local address 127.5.71.190:45817)
0923 20:49:53.232011 (+ 15224us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:53.232035 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:53.232104 (+ 69us) server_negotiation.cc:407] Connection header received
0923 20:49:53.232604 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:53.232633 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:53.233227 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:53.233743 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:53.236401 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.238615 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.241937 (+ 3322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.243363 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.243769 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:53.249430 (+ 5661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:53.249443 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:53.249468 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:53.249522 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:53.253725 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:53.254852 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:53.254863 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:53.254871 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:53.255092 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:53.255648 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:53.255654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:53.255659 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:53.256355 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:53.256551 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:53.257740 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0923 20:49:53.258190 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14870,"thread_start_us":17387,"threads_started":1}
I20260923 20:49:53.259719 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:53.215570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:40771)
0923 20:49:53.216683 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:53.216738 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:53.219163 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:53.234068 (+ 14905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:53.234086 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:53.235031 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:53.236077 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:53.236116 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.238897 (+ 2781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.238906 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:53.241648 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:53.241675 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.243749 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.243776 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:53.244215 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:53.245119 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:53.245166 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:53.249032 (+ 3866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:53.253937 (+ 4905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:53.253950 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:53.253970 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:53.254674 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:53.255233 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:53.255238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:53.255243 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:53.255510 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:53.256520 (+ 1010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:53.256532 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:53.257069 (+ 537us) client_negotiation.cc:769] Sending connection context
0923 20:49:53.257437 (+ 368us) client_negotiation.cc:240] Negotiation successful
0923 20:49:53.258159 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":696,"threads_started":1}
I20260923 20:49:53.263701 20047 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45817
I20260923 20:49:53.264083 20047 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:53.264992 20047 heartbeater.cc:511] Master 127.5.71.190:45817 requested a full tablet report, sending...
I20260923 20:49:53.267725 19862 ts_manager.cc:194] Registered new tserver with Master: 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927)
I20260923 20:49:53.269744 19862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:40771
W20260923 20:49:53.584550 20051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:53.584867 20051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:53.585021 20051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:53.592875 20051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:53.593035 20051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:53.593302 20051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:49:53.604326 20051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:40607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=46851
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:53.605870 20051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:53.607404 20051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:53.623605 20057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:53.623546 20056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:53.624238 20059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:53.627910 20051 server_base.cc:1034] running on GCE node
I20260923 20:49:53.628633 20051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:53.629805 20051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:53.631059 20051 hybrid_clock.cc:648] HybridClock initialized: now 1790196593630999 us; error 60 us; skew 500 ppm
I20260923 20:49:53.631436 20051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:53 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196593, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:53.635388 20051 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:49:53.637262 20051 webserver.cc:460] Webserver started at http://127.5.71.130:46851/ using document root <none> and password file <none>
I20260923 20:49:53.638103 20051 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:53.638286 20051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:53.644680 20051 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:49:53.648393 20066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:53.650228 20051 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260923 20:49:53.650460 20051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "db2f2ff1dd7949128c285a8c98295ae3"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "567144c5020c61e47ea317f4eaac4457"
server_key_iv: "0a2ce4f45737bee36a3a5ef3185cc514"
server_key_version: "encryptionkey@0"
I20260923 20:49:53.651612 20051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:53.671401 20051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:53.676208 20051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:53.676517 20051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:53.677827 20051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:53.679446 20051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:53.679608 20051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:53.679752 20051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:53.679891 20051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260923 20:49:53.715399 20051 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:40607
I20260923 20:49:53.715444 20178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:40607 every 8 connection(s)
I20260923 20:49:53.717207 20051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:49:53.720036 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20051
I20260923 20:49:53.720799 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:41749
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:45817
--webserver_port=38065
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:49:53 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196593, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:53.741490 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:53.720445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:43351 (local address 127.5.71.190:45817)
0923 20:49:53.720674 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:53.720681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:53.722164 (+ 1483us) server_negotiation.cc:407] Connection header received
0923 20:49:53.724274 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:53.724279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:53.724391 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:53.724506 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:53.726131 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.726906 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.728914 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.729757 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.729906 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:53.733612 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:53.733621 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:53.733628 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:53.733674 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:53.737446 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:53.738416 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:53.738425 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:53.738432 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:53.738616 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:53.739177 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:53.739185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:53.739191 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:53.739667 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:53.739891 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:53.740691 (+ 800us) server_negotiation.cc:299] Negotiation successful
0923 20:49:53.741016 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 20:49:53.742440 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:53.721037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:43351)
0923 20:49:53.721990 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:53.722038 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:53.723950 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:53.724795 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:53.724804 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:53.725331 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:53.725935 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:53.725953 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.727180 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.727184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:53.728690 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:53.728763 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:53.729988 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:53.729993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:53.730207 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:53.730967 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:53.730989 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:53.733347 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:53.737642 (+ 4295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:53.737651 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:53.737674 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:53.738224 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:53.738798 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:53.738802 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:53.738805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:53.739033 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:53.739868 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:53.739875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:53.740259 (+ 384us) client_negotiation.cc:769] Sending connection context
0923 20:49:53.740516 (+ 257us) client_negotiation.cc:240] Negotiation successful
0923 20:49:53.741068 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":419,"threads_started":1}
I20260923 20:49:53.744319 20179 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45817
I20260923 20:49:53.744688 20179 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:53.745522 20179 heartbeater.cc:511] Master 127.5.71.190:45817 requested a full tablet report, sending...
I20260923 20:49:53.747120 19862 ts_manager.cc:194] Registered new tserver with Master: db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
I20260923 20:49:53.748098 19862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:43351
W20260923 20:49:54.086206 20182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:54.086544 20182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:54.086691 20182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:54.096972 20182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:54.097124 20182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:54.097249 20182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:49:54.110867 20182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=38065
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:54.112483 20182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:54.114120 20182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:54.130205 20190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:54.130527 20187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:54.130578 20188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:54.131856 20182 server_base.cc:1034] running on GCE node
I20260923 20:49:54.132587 20182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:54.133693 20182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:54.134971 20182 hybrid_clock.cc:648] HybridClock initialized: now 1790196594134892 us; error 72 us; skew 500 ppm
I20260923 20:49:54.135471 20182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:54 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196594, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:54.139777 20182 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:49:54.141667 20182 webserver.cc:460] Webserver started at http://127.5.71.131:38065/ using document root <none> and password file <none>
I20260923 20:49:54.142499 20182 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:54.142633 20182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:54.149374 20182 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:49:54.152835 20197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:54.154901 20182 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:49:54.155224 20182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "f1c2072da61e4ae9a0c73a1811d17023"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "6957487e687919f1ef7b2edc943c9b45"
server_key_iv: "f453b34f52ec06a950d610dae8f458f6"
server_key_version: "encryptionkey@0"
I20260923 20:49:54.156414 20182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:54.190563 20182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:54.195854 20182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:54.196110 20182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:54.197165 20182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:54.198514 20182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:54.198609 20182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:54.198683 20182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:54.198743 20182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:54.238183 20182 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:41749
I20260923 20:49:54.238298 20309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:41749 every 8 connection(s)
I20260923 20:49:54.239964 20182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:49:54.250325 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20182
Sep 23 20:49:54 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196594, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:49:54.272469 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.244173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:56695 (local address 127.5.71.190:45817)
0923 20:49:54.244937 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.244943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.245895 (+ 952us) server_negotiation.cc:407] Connection header received
0923 20:49:54.248931 (+ 3036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.248938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.249058 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.249243 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:54.251631 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.252936 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.255271 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.256329 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.256467 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.263226 (+ 6759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:54.263234 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:54.263238 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:54.263303 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:54.266878 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.268078 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.268086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.268091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.268276 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.268987 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.268993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.268996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.269401 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:54.269502 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.270710 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.271910 (+ 1200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":634,"spinlock_wait_cycles":10368,"thread_start_us":490,"threads_started":1}
I20260923 20:49:54.273622 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.244486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:56695)
0923 20:49:54.245615 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.245672 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.247700 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.249528 (+ 1828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.249539 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.250150 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:54.251364 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.251403 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.253212 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.253217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.254962 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.254983 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.256801 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.256815 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.257358 (+ 543us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.258847 (+ 1489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:54.258901 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:54.262901 (+ 4000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:54.267124 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:54.267134 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:54.267152 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:54.267749 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:54.268467 (+ 718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:54.268470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:54.268473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:54.268838 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:54.269560 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:54.269572 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:54.270139 (+ 567us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.270507 (+ 368us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.271127 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":677,"threads_started":1}
I20260923 20:49:54.274236 20047 heartbeater.cc:503] Master 127.5.71.190:45817 was elected leader, sending a full tablet report...
I20260923 20:49:54.274681 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.251633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.5.71.190:45817)
0923 20:49:54.252592 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.252597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.252615 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:49:54.252664 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.252667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.252769 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.252875 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:54.254462 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.256256 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.259978 (+ 3722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.261561 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.261728 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.265608 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:54.265617 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:54.265622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:54.265692 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:54.269202 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.270518 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.270524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.270526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.270715 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.271653 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.271658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.271661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.272212 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:54.272515 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.273632 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.274229 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"spinlock_wait_cycles":1920,"thread_start_us":647,"threads_started":1}
I20260923 20:49:54.276648 20310 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45817
I20260923 20:49:54.277156 20310 heartbeater.cc:464] Registering TS with master...
I20260923 20:49:54.278446 20310 heartbeater.cc:511] Master 127.5.71.190:45817 requested a full tablet report, sending...
I20260923 20:49:54.281822 19861 ts_manager.cc:194] Registered new tserver with Master: f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749)
I20260923 20:49:54.284194 19861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:56695
I20260923 20:49:54.295558 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:49:54.307878 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.298097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.5.71.190:45817)
0923 20:49:54.298328 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.298333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.299132 (+ 799us) server_negotiation.cc:407] Connection header received
0923 20:49:54.299368 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.299372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.299578 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.299756 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:49:54.300786 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.301592 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.304192 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.305135 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.305268 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.306170 (+ 902us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:49:54.306924 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:49:54.307056 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.307448 (+ 392us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.307535 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:49:54.309137 19861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:49:54.317591 19861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:49:54.351809 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.334561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40607 (local address 127.0.0.1:59658)
0923 20:49:54.335965 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.335988 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.336133 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.338865 (+ 2732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.338880 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.338950 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.339953 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.339980 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.343384 (+ 3404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.343395 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.346937 (+ 3542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.346968 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.350023 (+ 3055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.350032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.350377 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.350472 (+ 95us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.350614 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.350973 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"thread_start_us":1090,"threads_started":1}
I20260923 20:49:54.352545 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.335370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.5.71.130:40607)
0923 20:49:54.337639 (+ 2269us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.337653 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.337704 (+ 51us) server_negotiation.cc:407] Connection header received
0923 20:49:54.337884 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.337896 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.338336 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.340415 (+ 2079us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.340832 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.343139 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.347563 (+ 4424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.349665 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.351437 (+ 1772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.351759 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.351916 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.352152 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1871,"thread_start_us":2320,"threads_started":1}
I20260923 20:49:54.358755 20114 tablet_service.cc:1467] Processing CreateTablet for tablet df171e0f50e54bc29c89c3c94198e028 (DEFAULT_TABLE table=test-table [id=8aac37a1d5474831b97d93d0d3c3658b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:54.360625 20114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df171e0f50e54bc29c89c3c94198e028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:54.360790 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.334488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.5.71.131:41749)
0923 20:49:54.350248 (+ 15760us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.350262 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.350308 (+ 46us) server_negotiation.cc:407] Connection header received
0923 20:49:54.350442 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.350453 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.350827 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.352177 (+ 1350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.352485 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.353783 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.357152 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.359390 (+ 2238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.359714 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.359988 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.360207 (+ 219us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.360426 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15513,"thread_start_us":16521,"threads_started":1}
I20260923 20:49:54.360855 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.334197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41749 (local address 127.0.0.1:45678)
0923 20:49:54.335283 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.335355 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.335601 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.351114 (+ 15513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.351119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.351153 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.351938 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.351959 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.353960 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.353966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.356902 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.356924 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.359717 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.359726 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.359941 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.359979 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.360116 (+ 137us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.360306 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":1,"thread_start_us":369,"threads_started":1}
I20260923 20:49:54.360950 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.336787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:42752)
0923 20:49:54.349625 (+ 12838us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.349666 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.349923 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.351428 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.351434 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.351601 (+ 167us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.352189 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.352203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.354891 (+ 2688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.354900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.357546 (+ 2646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.357572 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.360205 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.360211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.360369 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.360401 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.360485 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.360622 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12458,"thread_start_us":12230,"threads_started":1}
I20260923 20:49:54.361619 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.337418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.5.71.129:40927)
0923 20:49:54.348380 (+ 10962us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.348393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.350118 (+ 1725us) server_negotiation.cc:407] Connection header received
0923 20:49:54.350283 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.350295 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.350746 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.351047 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.352529 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.354649 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.357858 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.359589 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.359778 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.360042 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.361100 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.361297 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10651,"thread_start_us":11025,"threads_started":1}
I20260923 20:49:54.365433 20245 tablet_service.cc:1467] Processing CreateTablet for tablet df171e0f50e54bc29c89c3c94198e028 (DEFAULT_TABLE table=test-table [id=8aac37a1d5474831b97d93d0d3c3658b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:54.366006 19982 tablet_service.cc:1467] Processing CreateTablet for tablet df171e0f50e54bc29c89c3c94198e028 (DEFAULT_TABLE table=test-table [id=8aac37a1d5474831b97d93d0d3c3658b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:49:54.367128 20245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df171e0f50e54bc29c89c3c94198e028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:54.367486 19982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df171e0f50e54bc29c89c3c94198e028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:49:54.376117 20327 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Bootstrap starting.
I20260923 20:49:54.379878 20327 tablet_bootstrap.cc:654] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:54.381238 20327 log.cc:824] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:54.384294 20329 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Bootstrap starting.
I20260923 20:49:54.385296 20330 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Bootstrap starting.
I20260923 20:49:54.386652 20327 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: No bootstrap required, opened a new log
I20260923 20:49:54.387017 20327 ts_tablet_manager.cc:1397] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260923 20:49:54.388561 20329 tablet_bootstrap.cc:654] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:54.389425 20330 tablet_bootstrap.cc:654] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Neither blocks nor log segments found. Creating new log.
I20260923 20:49:54.389823 20329 log.cc:824] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:54.390726 20330 log.cc:824] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:54.392983 20329 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: No bootstrap required, opened a new log
I20260923 20:49:54.393329 20329 ts_tablet_manager.cc:1397] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260923 20:49:54.396377 20330 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: No bootstrap required, opened a new log
I20260923 20:49:54.396694 20330 ts_tablet_manager.cc:1397] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260923 20:49:54.398693 20327 raft_consensus.cc:348] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.399238 20327 raft_consensus.cc:374] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:54.399379 20327 raft_consensus.cc:729] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2f2ff1dd7949128c285a8c98295ae3, State: Initialized, Role: FOLLOWER
I20260923 20:49:54.400125 20327 consensus_queue.cc:260] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.401542 20179 heartbeater.cc:503] Master 127.5.71.190:45817 was elected leader, sending a full tablet report...
I20260923 20:49:54.402298 20327 ts_tablet_manager.cc:1428] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260923 20:49:54.401952 20329 raft_consensus.cc:348] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.402462 20329 raft_consensus.cc:374] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:54.402554 20329 raft_consensus.cc:729] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef2a69fc4bb442b8f7fe429b3def5c4, State: Initialized, Role: FOLLOWER
I20260923 20:49:54.403195 20329 consensus_queue.cc:260] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.405165 20329 ts_tablet_manager.cc:1428] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260923 20:49:54.407341 20330 raft_consensus.cc:348] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.407868 20330 raft_consensus.cc:374] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:49:54.407985 20330 raft_consensus.cc:729] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1c2072da61e4ae9a0c73a1811d17023, State: Initialized, Role: FOLLOWER
I20260923 20:49:54.408669 20330 consensus_queue.cc:260] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.409840 20310 heartbeater.cc:503] Master 127.5.71.190:45817 was elected leader, sending a full tablet report...
I20260923 20:49:54.410357 20330 ts_tablet_manager.cc:1428] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
W20260923 20:49:54.464463 20048 tablet.cc:2369] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:49:54.469491 20180 tablet.cc:2369] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:49:54.492787 20311 tablet.cc:2369] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:49:54.503427 20335 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:54.503779 20335 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.505745 20335 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607), 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927)
I20260923 20:49:54.508922 20333 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:54.509239 20333 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.510895 20333 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749), 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927)
I20260923 20:49:54.513996 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.506347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:53215 (local address 127.5.71.129:40927)
0923 20:49:54.506606 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.506614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.507108 (+ 494us) server_negotiation.cc:407] Connection header received
0923 20:49:54.507308 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.507312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.507371 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.507494 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.508075 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.509112 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.511547 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.512812 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.512942 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.513029 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.513531 (+ 502us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.513726 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260923 20:49:54.514283 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.505951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40607 (local address 127.5.71.131:35719)
0923 20:49:54.506308 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.506331 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.506469 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.506867 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.506872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.506909 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.507137 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.507145 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.508809 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.508816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.510872 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.510887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.513063 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.513072 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.513294 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.513329 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.513420 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.513570 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260923 20:49:54.514880 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.506188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:35719 (local address 127.5.71.130:40607)
0923 20:49:54.506472 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.506479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.506501 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:49:54.506587 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.506593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.506696 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.506813 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.507511 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.508647 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.511069 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.512421 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.512573 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.512653 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.514497 (+ 1844us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.514639 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260923 20:49:54.514842 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.506269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.5.71.131:53215)
0923 20:49:54.507008 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.507022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.507165 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.507522 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.507525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.507545 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.507934 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.507943 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.509289 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.509295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.511366 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.511379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.513197 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.513203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.513355 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.513398 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.514305 (+ 907us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.514464 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":441,"threads_started":1}
I20260923 20:49:54.516202 20002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df171e0f50e54bc29c89c3c94198e028" candidate_uuid: "f1c2072da61e4ae9a0c73a1811d17023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" is_pre_election: true
I20260923 20:49:54.516908 20002 raft_consensus.cc:2454] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1c2072da61e4ae9a0c73a1811d17023 in term 0.
I20260923 20:49:54.517925 20198 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4, f1c2072da61e4ae9a0c73a1811d17023; no voters:
I20260923 20:49:54.518115 20134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df171e0f50e54bc29c89c3c94198e028" candidate_uuid: "f1c2072da61e4ae9a0c73a1811d17023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2f2ff1dd7949128c285a8c98295ae3" is_pre_election: true
I20260923 20:49:54.518486 20335 raft_consensus.cc:2781] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:49:54.518630 20335 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:49:54.518745 20335 raft_consensus.cc:3037] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:54.518668 20134 raft_consensus.cc:2454] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1c2072da61e4ae9a0c73a1811d17023 in term 0.
I20260923 20:49:54.522269 20335 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.523682 20134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df171e0f50e54bc29c89c3c94198e028" candidate_uuid: "f1c2072da61e4ae9a0c73a1811d17023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2f2ff1dd7949128c285a8c98295ae3"
I20260923 20:49:54.523702 20002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df171e0f50e54bc29c89c3c94198e028" candidate_uuid: "f1c2072da61e4ae9a0c73a1811d17023" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4"
I20260923 20:49:54.523932 20134 raft_consensus.cc:3037] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:54.523939 20002 raft_consensus.cc:3037] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:49:54.525741 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.516645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:42207 (local address 127.5.71.129:40927)
0923 20:49:54.516816 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.516824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.517594 (+ 770us) server_negotiation.cc:407] Connection header received
0923 20:49:54.517882 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.517888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.517967 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.518034 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.519103 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.520250 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.522931 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.524368 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.524537 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.524638 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.525260 (+ 622us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.525422 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:49:54.526131 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.512782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:42167 (local address 127.5.71.131:41749)
0923 20:49:54.512984 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.512989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.515732 (+ 2743us) server_negotiation.cc:407] Connection header received
0923 20:49:54.516060 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.516065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.516137 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.516214 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:49:54.520144 (+ 3930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.521314 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.523768 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.525003 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.525124 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.525211 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.525773 (+ 562us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.525917 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:49:54.526530 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.516105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.5.71.130:42207)
0923 20:49:54.517484 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.517513 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.517689 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.518280 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.518288 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.518320 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.518914 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.518931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.520410 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.520416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.522657 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.522671 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.524857 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.524863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.525062 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.525102 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.525968 (+ 866us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.526105 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1231,"mutex_wait_us":1,"thread_start_us":1015,"threads_started":1}
I20260923 20:49:54.526758 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.511500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41749 (local address 127.5.71.130:42167)
0923 20:49:54.515632 (+ 4132us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.515654 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:54.515882 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:54.516420 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:54.516425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:54.516465 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:49:54.516837 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.516849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.521502 (+ 4653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.521507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.523583 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:54.523598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.525402 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.525407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:54.525575 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.525607 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:49:54.526290 (+ 683us) client_negotiation.cc:240] Negotiation successful
0923 20:49:54.526423 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3902,"thread_start_us":3711,"threads_started":1}
I20260923 20:49:54.527276 20002 raft_consensus.cc:2454] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1c2072da61e4ae9a0c73a1811d17023 in term 1.
I20260923 20:49:54.527639 20335 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [CANDIDATE]: Term 1 election: Requested vote from peers db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607), 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927)
I20260923 20:49:54.527449 20001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df171e0f50e54bc29c89c3c94198e028" candidate_uuid: "db2f2ff1dd7949128c285a8c98295ae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" is_pre_election: true
I20260923 20:49:54.527743 20198 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4, f1c2072da61e4ae9a0c73a1811d17023; no voters:
I20260923 20:49:54.527997 20001 raft_consensus.cc:2379] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db2f2ff1dd7949128c285a8c98295ae3 in current term 1: Already voted for candidate f1c2072da61e4ae9a0c73a1811d17023 in this term.
I20260923 20:49:54.528116 20335 raft_consensus.cc:2781] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:49:54.528741 20265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df171e0f50e54bc29c89c3c94198e028" candidate_uuid: "db2f2ff1dd7949128c285a8c98295ae3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1c2072da61e4ae9a0c73a1811d17023" is_pre_election: true
I20260923 20:49:54.529309 20134 raft_consensus.cc:2454] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1c2072da61e4ae9a0c73a1811d17023 in term 1.
I20260923 20:49:54.529397 20335 raft_consensus.cc:686] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 LEADER]: Becoming Leader. State: Replica: f1c2072da61e4ae9a0c73a1811d17023, State: Running, Role: LEADER
I20260923 20:49:54.529995 20335 consensus_queue.cc:237] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:54.533967 20069 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2f2ff1dd7949128c285a8c98295ae3; no voters: 6ef2a69fc4bb442b8f7fe429b3def5c4, f1c2072da61e4ae9a0c73a1811d17023
I20260923 20:49:54.534461 20333 raft_consensus.cc:2726] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 20:49:54.535368 19862 catalog_manager.cc:5495] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "f1c2072da61e4ae9a0c73a1811d17023" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } health_report { overall_health: UNKNOWN } } }
I20260923 20:49:54.589905 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.584007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.5.71.131:41749)
0923 20:49:54.584219 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.584228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.584250 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:49:54.584511 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.584516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.584615 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.584791 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:49:54.585208 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.586037 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.587431 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.587970 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.588118 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.588694 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:49:54.589241 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:49:54.589419 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.589473 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.589561 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 20:49:54.599707 20134 raft_consensus.cc:1261] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Refusing update from remote peer f1c2072da61e4ae9a0c73a1811d17023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:54.599714 20001 raft_consensus.cc:1261] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Refusing update from remote peer f1c2072da61e4ae9a0c73a1811d17023: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:49:54.600772 20339 consensus_queue.cc:1035] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:54.601172 20335 consensus_queue.cc:1035] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:49:54.615636 20342 mvcc.cc:205] Tried to move back new op lower bound from 7332645251467853824 to 7332645251198889984. Current Snapshot: MvccSnapshot[applied={T|T < 7332645251467853824}]
I20260923 20:49:54.620134 20344 mvcc.cc:205] Tried to move back new op lower bound from 7332645251467853824 to 7332645251198889984. Current Snapshot: MvccSnapshot[applied={T|T < 7332645251467853824}]
I20260923 20:49:54.640403 19861 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:49:54.640740 19861 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:49:54.644873 19861 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Sending DeleteTablet for 3 replicas of tablet df171e0f50e54bc29c89c3c94198e028
I20260923 20:49:54.646080 20245 tablet_service.cc:1514] Processing DeleteTablet for tablet df171e0f50e54bc29c89c3c94198e028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:54 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:45678
I20260923 20:49:54.646224 19982 tablet_service.cc:1514] Processing DeleteTablet for tablet df171e0f50e54bc29c89c3c94198e028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:54 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:42752
I20260923 20:49:54.646575 20114 tablet_service.cc:1514] Processing DeleteTablet for tablet df171e0f50e54bc29c89c3c94198e028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:49:54 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:59658
I20260923 20:49:54.647993 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19899
I20260923 20:49:54.657682 20354 tablet_replica.cc:330] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: stopping tablet replica
I20260923 20:49:54.658540 20354 raft_consensus.cc:2229] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:49:54.659385 20353 tablet_replica.cc:330] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: stopping tablet replica
I20260923 20:49:54.660223 20354 raft_consensus.cc:2258] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:54.660921 20353 raft_consensus.cc:2229] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:49:54.661981 20353 raft_consensus.cc:2258] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:54.661993 20352 tablet_replica.cc:330] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: stopping tablet replica
I20260923 20:49:54.662423 20352 raft_consensus.cc:2229] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:49:54.663187 20352 raft_consensus.cc:2258] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:49:54.664425 20354 ts_tablet_manager.cc:1905] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:54.666410 20353 ts_tablet_manager.cc:1905] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:54.668498 20352 ts_tablet_manager.cc:1905] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:49:54.672200 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.649789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.5.71.129:40927)
0923 20:49:54.650015 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.650024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.650046 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:49:54.650130 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.650138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.650257 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.650374 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:54.653069 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.654223 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.656722 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.657809 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.657971 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.662952 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:54.662966 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:54.662987 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:54.663045 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:54.667422 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.668537 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.668548 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.668553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.668732 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.669282 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.669291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.669298 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.669783 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:54.670175 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.671504 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.671761 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:49:54.672588 20354 ts_tablet_manager.cc:1918] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:54.672802 20354 log.cc:1196] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/df171e0f50e54bc29c89c3c94198e028
I20260923 20:49:54.673167 20042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:54.673595 20354 ts_tablet_manager.cc:1939] T df171e0f50e54bc29c89c3c94198e028 P db2f2ff1dd7949128c285a8c98295ae3: Deleting consensus metadata
I20260923 20:49:54.676438 19847 catalog_manager.cc:4841] TS db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): tablet df171e0f50e54bc29c89c3c94198e028 (table test-table [id=8aac37a1d5474831b97d93d0d3c3658b]) successfully deleted
I20260923 20:49:54.677271 20352 ts_tablet_manager.cc:1918] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:49:54.677528 20352 log.cc:1196] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/df171e0f50e54bc29c89c3c94198e028
I20260923 20:49:54.678260 20352 ts_tablet_manager.cc:1939] T df171e0f50e54bc29c89c3c94198e028 P f1c2072da61e4ae9a0c73a1811d17023: Deleting consensus metadata
I20260923 20:49:54.680944 19848 catalog_manager.cc:4841] TS f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): tablet df171e0f50e54bc29c89c3c94198e028 (table test-table [id=8aac37a1d5474831b97d93d0d3c3658b]) successfully deleted
I20260923 20:49:54.784013 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19899
W20260923 20:49:54.802058 19846 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:40927 (error 108)
I20260923 20:49:54.802527 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 20051
I20260923 20:49:54.804141 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.803583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56654)
0923 20:49:54.803776 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.803958 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260923 20:49:54.804450 19846 catalog_manager.cc:4584] TS 6ef2a69fc4bb442b8f7fe429b3def5c4 (127.5.71.129:40927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df171e0f50e54bc29c89c3c94198e028: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
I20260923 20:49:54.818776 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.803276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.5.71.130:40607)
0923 20:49:54.803455 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.803460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.803480 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:49:54.803680 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.803687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.803778 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.803917 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:54.805580 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.806416 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.807770 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.808413 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.808548 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.811974 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:54.811991 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:54.812009 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:54.812050 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:54.815177 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.816258 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.816267 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.816269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.816441 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.816891 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.816895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.816898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.817348 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:54.817591 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.818010 (+ 419us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.818348 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 20:49:54.819495 20174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:54.858234 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.857652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56666)
0923 20:49:54.857902 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.858032 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260923 20:49:54.859457 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.858998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56680)
0923 20:49:54.859159 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.859250 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 20:49:54.912729 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 20051
I20260923 20:49:54.920037 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.919426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56690)
0923 20:49:54.919673 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.919826 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260923 20:49:54.921368 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.920982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56694)
0923 20:49:54.921144 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:54.921228 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260923 20:49:54.935731 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 20182
I20260923 20:49:54.951355 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:54.937119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37700 (local address 127.5.71.131:41749)
0923 20:49:54.937339 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:54.937347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:54.937393 (+ 46us) server_negotiation.cc:407] Connection header received
0923 20:49:54.937498 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:54.937502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:54.937563 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:54.937693 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:54.939097 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.940070 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.941465 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:54.941914 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:54.942082 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:54.944938 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:54.944949 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:54.944966 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:54.945007 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:54.948271 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.949084 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.949096 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.949098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.949258 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:54.949666 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:54.949670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:54.949676 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:54.950057 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:54.950396 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:54.950746 (+ 350us) server_negotiation.cc:299] Negotiation successful
0923 20:49:54.950994 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 20:49:54.952317 20305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:55.002573 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:55.001847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56702)
0923 20:49:55.002223 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:55.002357 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260923 20:49:55.004021 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:55.003506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:40927 (local address 127.0.0.1:56718)
0923 20:49:55.003728 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:55.003857 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:40927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260923 20:49:55.080943 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 20182
I20260923 20:49:55.102311 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 19829
I20260923 20:49:55.103729 19892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:49:55.220757 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 19829
I20260923 20:49:55.243957 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45817
--webserver_interface=127.5.71.190
--webserver_port=38819
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:55.592962 20360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:55.593289 20360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:55.593436 20360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:55.601034 20360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:49:55.601161 20360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:55.601207 20360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:55.601255 20360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:49:55.601362 20360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:49:55.611639 20360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=38819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:55.613190 20360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:55.614552 20360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:55.626816 20365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:55.626843 20368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:55.626892 20366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:55.628616 20360 server_base.cc:1034] running on GCE node
I20260923 20:49:55.629712 20360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:55.631366 20360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:55.632766 20360 hybrid_clock.cc:648] HybridClock initialized: now 1790196595632726 us; error 61 us; skew 500 ppm
I20260923 20:49:55.633230 20360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:55 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196595, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:55.636977 20360 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:49:55.638638 20360 webserver.cc:460] Webserver started at http://127.5.71.190:38819/ using document root <none> and password file <none>
I20260923 20:49:55.639457 20360 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:55.639611 20360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:55.645869 20360 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260923 20:49:55.651852 20375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:55.653571 20360 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260923 20:49:55.653883 20360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "96f65377c9024acfbecc0c020ecc806c"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "bfc22df0f765267ad615db92d40f8f2e"
server_key_iv: "78ff449aa28fee0b80dfda7b0e836f79"
server_key_version: "encryptionkey@0"
I20260923 20:49:55.655085 20360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:55.675738 20360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:55.680519 20360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:55.680962 20360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:55.703655 20360 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45817
I20260923 20:49:55.703682 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45817 every 8 connection(s)
I20260923 20:49:55.705420 20360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:49:55.709911 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20360
I20260923 20:49:55.710948 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:40927
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:45817
--webserver_port=46449
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:55.714133 20427 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:49:55.719563 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap starting.
I20260923 20:49:55.741060 20427 log.cc:824] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:55.759685 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:49:55.760335 20427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c: Bootstrap complete.
I20260923 20:49:55.770124 20427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:55.771426 20427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f65377c9024acfbecc0c020ecc806c, State: Initialized, Role: FOLLOWER
I20260923 20:49:55.772145 20427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:55.772396 20427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:49:55.772554 20427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:49:55.772840 20427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 2 FOLLOWER]: Advancing to term 3
I20260923 20:49:55.776865 20427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:55.777395 20427 leader_election.cc:302] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96f65377c9024acfbecc0c020ecc806c; no voters:
I20260923 20:49:55.778126 20427 leader_election.cc:288] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 20:49:55.778283 20431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 3 FOLLOWER]: Leader election won for term 3
I20260923 20:49:55.779490 20431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [term 3 LEADER]: Becoming Leader. State: Replica: 96f65377c9024acfbecc0c020ecc806c, State: Running, Role: LEADER
I20260923 20:49:55.780224 20431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } }
I20260923 20:49:55.780503 20427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:49:55.783620 20432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "96f65377c9024acfbecc0c020ecc806c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } } }
I20260923 20:49:55.784134 20432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:55.783550 20433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f65377c9024acfbecc0c020ecc806c. Latest consensus state: current_term: 3 leader_uuid: "96f65377c9024acfbecc0c020ecc806c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f65377c9024acfbecc0c020ecc806c" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45817 } } }
I20260923 20:49:55.784396 20433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f65377c9024acfbecc0c020ecc806c [sys.catalog]: This master's current role is: LEADER
I20260923 20:49:55.797250 20436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:49:55.806632 20436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:49:55.808162 20436 catalog_manager.cc:1259] Loaded cluster ID: a37f6dea967d4382ae70cef140509556
I20260923 20:49:55.808269 20436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 20:49:55.810232 20436 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc206f979ad google::LogMessage::Fail() at ??:0
@ 0x7fc206f9a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc206f974de google::LogMessage::Flush() at ??:0
@ 0x7fc206f98a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc21c9ca512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc21c9c7976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc21ca4f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc21ca4f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc209be3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc209be691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc209be6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc209bc1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc20ce9b6db start_thread at ??:0
@ 0x7fc205e3161f clone at ??:0
W20260923 20:49:56.136677 20429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:56.136974 20429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:56.137130 20429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:56.144761 20429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:56.144896 20429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:56.145081 20429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:49:56.155464 20429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:40927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=46449
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:56.156875 20429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:56.158226 20429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:56.172181 20469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:56.172134 20471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:56.172400 20468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:56.172953 20429 server_base.cc:1034] running on GCE node
I20260923 20:49:56.173593 20429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:56.174640 20429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:56.175930 20429 hybrid_clock.cc:648] HybridClock initialized: now 1790196596175855 us; error 49 us; skew 500 ppm
I20260923 20:49:56.176339 20429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:56 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196596, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:56.179852 20429 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:49:56.181413 20429 webserver.cc:460] Webserver started at http://127.5.71.129:46449/ using document root <none> and password file <none>
I20260923 20:49:56.182148 20429 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:56.182258 20429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:56.187899 20429 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260923 20:49:56.190896 20478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:56.192519 20429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:49:56.192678 20429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4"
format_stamp: "Formatted at 2026-09-23 20:49:49 on dist-test-slave-dn29"
server_key: "92e70ced720b74c1ba9024c8fa27c897"
server_key_iv: "823cfa8c9bbb4d9b8b3be3628b0c3f74"
server_key_version: "encryptionkey@0"
I20260923 20:49:56.193809 20429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:56.211026 20429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:56.215495 20429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:56.215845 20429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:56.216884 20429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:56.218894 20485 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260923 20:49:56.226148 20429 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 20:49:56.226290 20429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260923 20:49:56.226469 20429 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 20:49:56.229959 20429 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 20:49:56.230050 20429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260923 20:49:56.230621 20485 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Bootstrap starting.
I20260923 20:49:56.240386 20485 log.cc:824] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Log is configured to *not* fsync() on all Append() calls
I20260923 20:49:56.255136 20485 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:49:56.256080 20485 tablet_bootstrap.cc:492] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Bootstrap complete.
I20260923 20:49:56.256767 20485 ts_tablet_manager.cc:1397] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Time spent bootstrapping tablet: real 0.027s user 0.013s sys 0.012s
I20260923 20:49:56.272007 20485 raft_consensus.cc:348] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:56.274072 20485 raft_consensus.cc:729] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef2a69fc4bb442b8f7fe429b3def5c4, State: Initialized, Role: FOLLOWER
I20260923 20:49:56.275087 20485 consensus_queue.cc:260] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:56.276080 20429 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:40927
I20260923 20:49:56.276172 20592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:40927 every 8 connection(s)
I20260923 20:49:56.277325 20485 ts_tablet_manager.cc:1428] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260923 20:49:56.277685 20429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:49:56.279724 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20429
I20260923 20:49:56.280402 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:40607
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:45817
--webserver_port=46851
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:49:56.282972 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:56.280761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:51647)
0923 20:49:56.281772 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:56.282063 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":436,"threads_started":1}
W20260923 20:49:56.283721 20594 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45817 (0 consecutive failures): Network error: Failed to ping master at 127.5.71.190:45817: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
I20260923 20:49:56.284646 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:56.284295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:37763)
0923 20:49:56.284456 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:56.284540 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260923 20:49:56.285602 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:56.285244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:52591)
0923 20:49:56.285391 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:56.285477 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260923 20:49:56.285910 20594 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 20:49:56.631408 20597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:56.631759 20597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:56.631958 20597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:56.640031 20597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:56.640174 20597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:56.640403 20597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:49:56.650838 20597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:40607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=46851
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:56.652405 20597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:56.653807 20597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:56.672516 20602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:56.672447 20605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:56.672480 20603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:56.673791 20597 server_base.cc:1034] running on GCE node
I20260923 20:49:56.674503 20597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:56.675668 20597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:56.676923 20597 hybrid_clock.cc:648] HybridClock initialized: now 1790196596676867 us; error 43 us; skew 500 ppm
I20260923 20:49:56.677307 20597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:56 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196596, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:56.681576 20597 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:49:56.683488 20597 webserver.cc:460] Webserver started at http://127.5.71.130:46851/ using document root <none> and password file <none>
I20260923 20:49:56.684324 20597 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:56.684456 20597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:56.690472 20597 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260923 20:49:56.693590 20612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:56.695178 20597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:49:56.695389 20597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "db2f2ff1dd7949128c285a8c98295ae3"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "567144c5020c61e47ea317f4eaac4457"
server_key_iv: "0a2ce4f45737bee36a3a5ef3185cc514"
server_key_version: "encryptionkey@0"
I20260923 20:49:56.696650 20597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:56.713697 20597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:56.718582 20597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:56.718888 20597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:56.720126 20597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:56.721632 20597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:56.721760 20597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:56.721900 20597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:56.721992 20597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:56.756145 20597 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:40607
I20260923 20:49:56.756224 20724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:40607 every 8 connection(s)
I20260923 20:49:56.757709 20597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:49:56.763037 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:56.760883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:60675)
0923 20:49:56.761761 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:56.762008 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":327,"threads_started":1}
W20260923 20:49:56.763898 20725 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45817 (0 consecutive failures): Network error: Failed to ping master at 127.5.71.190:45817: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
I20260923 20:49:56.764813 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:56.764405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:41149)
0923 20:49:56.764587 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:56.764677 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260923 20:49:56.765756 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:56.765378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:35747)
0923 20:49:56.765536 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:56.765627 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260923 20:49:56.766006 20725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 20:49:56.767652 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20597
I20260923 20:49:56.768354 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:41749
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:45817
--webserver_port=38065
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:49:57.122582 20728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:49:57.122987 20728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:49:57.123172 20728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:49:57.130697 20728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:49:57.130875 20728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:49:57.131067 20728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:49:57.141312 20728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=38065
--tserver_master_addrs=127.5.71.190:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:49:57.142746 20728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:49:57.144245 20728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:49:57.158792 20733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:57.159340 20734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:49:57.159765 20736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:49:57.160557 20728 server_base.cc:1034] running on GCE node
I20260923 20:49:57.161243 20728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:49:57.162315 20728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:49:57.163553 20728 hybrid_clock.cc:648] HybridClock initialized: now 1790196597163438 us; error 112 us; skew 500 ppm
I20260923 20:49:57.163968 20728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:49:57 dist-test-slave-dn29 krb5kdc[19276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196597, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:49:57.167621 20728 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:49:57.169292 20728 webserver.cc:460] Webserver started at http://127.5.71.131:38065/ using document root <none> and password file <none>
I20260923 20:49:57.170046 20728 fs_manager.cc:359] Metadata directory not provided
I20260923 20:49:57.170197 20728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:49:57.175951 20728 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260923 20:49:57.178727 20743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:57.180210 20728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:49:57.180411 20728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "f1c2072da61e4ae9a0c73a1811d17023"
format_stamp: "Formatted at 2026-09-23 20:49:50 on dist-test-slave-dn29"
server_key: "6957487e687919f1ef7b2edc943c9b45"
server_key_iv: "f453b34f52ec06a950d610dae8f458f6"
server_key_version: "encryptionkey@0"
I20260923 20:49:57.181425 20728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:49:57.207062 20728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:49:57.211266 20728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:49:57.211570 20728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:49:57.212757 20728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:49:57.214150 20728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:49:57.214294 20728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:57.214418 20728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:49:57.214505 20728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:49:57.249832 20728 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:41749
I20260923 20:49:57.249927 20855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:41749 every 8 connection(s)
I20260923 20:49:57.251407 20728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:49:57.254756 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20728
I20260923 20:49:57.256675 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.254486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:57251)
0923 20:49:57.255414 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.255670 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":374,"threads_started":1}
W20260923 20:49:57.257577 20856 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45817 (0 consecutive failures): Network error: Failed to ping master at 127.5.71.190:45817: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
I20260923 20:49:57.258397 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.258034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:47155)
0923 20:49:57.258194 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.258284 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260923 20:49:57.259239 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.258920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:32801)
0923 20:49:57.259042 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.259115 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260923 20:49:57.259552 20856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 20:49:57.287318 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.286516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:33095)
0923 20:49:57.286983 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.287080 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":219,"threads_started":1}
I20260923 20:49:57.411887 20861 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:57.412295 20861 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:57.414218 20861 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749), db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
Sep 23 20:49:57 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196596, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
Sep 23 20:49:57 dist-test-slave-dn29 krb5kdc[19276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196596, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:49:57.439273 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.414657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:41749 (local address 127.5.71.129:37593)
0923 20:49:57.418210 (+ 3553us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.418232 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:57.418360 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:57.421364 (+ 3004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:57.421373 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:57.422397 (+ 1024us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:57.422661 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:57.422671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.424477 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.424481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:57.425673 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:57.425686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.427076 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.427083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:57.427254 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:57.427890 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:57.427913 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:57.430609 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:57.435006 (+ 4397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:57.435012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:57.435017 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:57.435847 (+ 830us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:57.436376 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:57.436381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:57.436384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:57.436632 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:57.437482 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:57.437487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:57.437903 (+ 416us) client_negotiation.cc:769] Sending connection context
0923 20:49:57.438179 (+ 276us) client_negotiation.cc:240] Negotiation successful
0923 20:49:57.438787 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3362,"mutex_wait_us":121,"thread_start_us":3178,"threads_started":1}
I20260923 20:49:57.439654 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.414964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:37593 (local address 127.5.71.131:41749)
0923 20:49:57.418632 (+ 3668us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:57.418643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:57.418681 (+ 38us) server_negotiation.cc:407] Connection header received
0923 20:49:57.420526 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:57.420537 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:57.420977 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:57.421330 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:57.422860 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.424275 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.425841 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.426645 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.427093 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:57.431093 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:57.431107 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:57.431123 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:57.431168 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:57.434827 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:57.436028 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:57.436037 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:57.436045 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:57.436239 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:57.436753 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:57.436757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:57.436759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:57.437309 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:57.437544 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:57.438435 (+ 891us) server_negotiation.cc:299] Negotiation successful
0923 20:49:57.438845 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3511,"thread_start_us":3334,"threads_started":1}
I20260923 20:49:57.439378 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.414557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:40607 (local address 127.5.71.129:43605)
0923 20:49:57.414915 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.414968 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 20:49:57.415659 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:49:57.419981 (+ 4322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:49:57.419992 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:49:57.420795 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:49:57.421863 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:57.421897 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.423557 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.423562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:57.424953 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:49:57.424966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.426043 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.426049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:49:57.426319 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:57.427206 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:49:57.427239 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:49:57.430478 (+ 3239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:49:57.434826 (+ 4348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:57.434838 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:57.434854 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:57.435807 (+ 953us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:57.436392 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:49:57.436396 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:49:57.436398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:49:57.436619 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:49:57.437476 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:49:57.437486 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:49:57.437985 (+ 499us) client_negotiation.cc:769] Sending connection context
0923 20:49:57.438167 (+ 182us) client_negotiation.cc:240] Negotiation successful
0923 20:49:57.438960 (+ 793us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":262,"spinlock_wait_cycles":21760}
I20260923 20:49:57.439296 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.414951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:43605 (local address 127.5.71.130:40607)
0923 20:49:57.416586 (+ 1635us) server_negotiation.cc:206] Beginning negotiation
0923 20:49:57.416603 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:49:57.416648 (+ 45us) server_negotiation.cc:407] Connection header received
0923 20:49:57.418962 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:49:57.418973 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:49:57.419424 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:49:57.419742 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:49:57.422133 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.423355 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.425124 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:49:57.425818 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:49:57.426039 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:49:57.430892 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:49:57.430902 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:49:57.430920 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:49:57.431045 (+ 125us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:49:57.434617 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:57.436022 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:57.436033 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:57.436042 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:57.436273 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:49:57.436750 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:49:57.436755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:49:57.436758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:49:57.437306 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:49:57.437549 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0923 20:49:57.438432 (+ 883us) server_negotiation.cc:299] Negotiation successful
0923 20:49:57.438843 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1427,"thread_start_us":1293,"threads_started":1}
W20260923 20:49:57.442919 20481 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
W20260923 20:49:57.443722 20480 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
I20260923 20:49:57.443985 20480 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4; no voters: db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023
I20260923 20:49:57.444468 20861 raft_consensus.cc:2726] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:49:57.767506 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:57.766630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:46055)
0923 20:49:57.767128 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:57.767272 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":228,"threads_started":1}
I20260923 20:49:58.261282 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:58.260275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:60179)
0923 20:49:58.260796 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:58.260951 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":309,"threads_started":1}
I20260923 20:49:58.289117 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:58.288346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:59461)
0923 20:49:58.288862 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:58.288945 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":292,"threads_started":1}
I20260923 20:49:58.769368 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:58.768631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:34835)
0923 20:49:58.769108 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:58.769195 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":238,"threads_started":1}
I20260923 20:49:59.091747 20869 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:49:59.092090 20869 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:49:59.092927 20869 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749), db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
W20260923 20:49:59.094028 20481 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
W20260923 20:49:59.094242 20480 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
I20260923 20:49:59.094399 20480 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4; no voters: db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023
I20260923 20:49:59.094697 20869 raft_consensus.cc:2726] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:49:59.263226 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:59.262277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:57349)
0923 20:49:59.262848 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:59.262972 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":334,"threads_started":1}
I20260923 20:49:59.290953 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:59.290087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:57321)
0923 20:49:59.290662 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:59.290761 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":267,"threads_started":1}
I20260923 20:49:59.771198 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:49:59.770404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:45993)
0923 20:49:59.770905 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0923 20:49:59.770995 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":252,"threads_started":1}
I20260923 20:50:00.265122 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:00.264325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:46651)
0923 20:50:00.264839 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:00.264943 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":238,"threads_started":1}
I20260923 20:50:00.293256 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:00.292166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:34099)
0923 20:50:00.292908 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:00.293003 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":357,"threads_started":1}
I20260923 20:50:00.773193 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:00.772357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:58235)
0923 20:50:00.772930 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:00.773011 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":326,"threads_started":1}
I20260923 20:50:01.267104 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:01.266290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:33717)
0923 20:50:01.266839 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:01.266927 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":304,"threads_started":1}
I20260923 20:50:01.295397 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:01.294423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:51791)
0923 20:50:01.295111 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:01.295213 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":413,"threads_started":1}
I20260923 20:50:01.774978 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:01.774210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:36743)
0923 20:50:01.774675 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:01.774808 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":239,"threads_started":1}
I20260923 20:50:01.912187 20879 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:50:01.912379 20879 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:50:01.913170 20879 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749), db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
W20260923 20:50:01.914620 20481 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
W20260923 20:50:01.914927 20480 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
I20260923 20:50:01.915050 20480 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4; no voters: db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023
I20260923 20:50:01.915381 20879 raft_consensus.cc:2726] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:50:02.269390 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:02.268577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:55835)
0923 20:50:02.269133 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:02.269229 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":320,"threads_started":1}
I20260923 20:50:02.297585 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:02.296517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:56643)
0923 20:50:02.297208 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:02.297332 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":382,"threads_started":1}
I20260923 20:50:02.777179 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:02.776150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:46377)
0923 20:50:02.776864 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:02.776965 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"spinlock_wait_cycles":29056,"thread_start_us":471,"threads_started":1}
I20260923 20:50:03.271512 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:03.270607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:39305)
0923 20:50:03.271179 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:03.271273 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":293,"threads_started":1}
I20260923 20:50:03.299476 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:03.298675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:33785)
0923 20:50:03.299187 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:03.299281 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":260,"threads_started":1}
I20260923 20:50:03.779470 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:03.778497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:55229)
0923 20:50:03.779142 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:03.779246 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":329,"threads_started":1}
I20260923 20:50:04.199146 20886 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:50:04.199548 20886 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:50:04.200538 20886 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749), db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
W20260923 20:50:04.201782 20480 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
W20260923 20:50:04.201984 20481 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
I20260923 20:50:04.202131 20481 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4; no voters: db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023
I20260923 20:50:04.202422 20886 raft_consensus.cc:2726] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:50:04.273718 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:04.272789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:42577)
0923 20:50:04.273389 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:04.273476 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":298,"threads_started":1}
I20260923 20:50:04.301628 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:04.300593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:34455)
0923 20:50:04.301319 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:04.301416 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":396,"threads_started":1}
I20260923 20:50:04.781769 20889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:04.780608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:59333)
0923 20:50:04.781375 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:04.781476 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":306,"threads_started":1}
I20260923 20:50:05.275583 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:05.274801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:38489)
0923 20:50:05.275278 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:05.275392 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":245,"threads_started":1}
I20260923 20:50:05.303953 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:05.302807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:50597)
0923 20:50:05.303459 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:05.303614 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":402,"threads_started":1}
I20260923 20:50:05.783870 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:05.782910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:36383)
0923 20:50:05.783511 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:05.783627 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":271,"threads_started":1}
I20260923 20:50:06.277556 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:06.276783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:52277)
0923 20:50:06.277313 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:06.277403 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":280,"threads_started":1}
I20260923 20:50:06.306313 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:06.305362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:60085)
0923 20:50:06.306024 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:06.306130 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":340,"threads_started":1}
I20260923 20:50:06.785830 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:06.784967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:32849)
0923 20:50:06.785529 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:06.785633 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
I20260923 20:50:07.279462 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:07.278633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:35057)
0923 20:50:07.279202 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:07.279291 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":351,"threads_started":1}
I20260923 20:50:07.308301 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:07.307386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:45469)
0923 20:50:07.308002 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:07.308107 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":260,"threads_started":1}
I20260923 20:50:07.787744 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:07.786881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:56685)
0923 20:50:07.787439 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:07.787535 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":280,"threads_started":1}
I20260923 20:50:08.281514 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:08.280576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:35349)
0923 20:50:08.281222 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:08.281352 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":359,"threads_started":1}
I20260923 20:50:08.310467 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:08.309396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:36845)
0923 20:50:08.310014 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:08.310145 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":289,"threads_started":1}
I20260923 20:50:08.789669 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:08.788898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:40821)
0923 20:50:08.789400 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:08.789491 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":291,"threads_started":1}
I20260923 20:50:09.283497 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:09.282631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:52487)
0923 20:50:09.283222 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:09.283311 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":266,"threads_started":1}
I20260923 20:50:09.312839 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:09.311877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:52487)
0923 20:50:09.312515 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:09.312635 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":360,"threads_started":1}
I20260923 20:50:09.791612 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:09.790820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:41157)
0923 20:50:09.791346 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:09.791443 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":269,"threads_started":1}
I20260923 20:50:10.285797 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:10.284802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:51305)
0923 20:50:10.285368 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:10.285504 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":245,"threads_started":1}
I20260923 20:50:10.314967 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:10.313974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:32937)
0923 20:50:10.314555 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:10.314744 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":331,"threads_started":1}
I20260923 20:50:10.793516 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:10.792708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:43423)
0923 20:50:10.793186 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:10.793302 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":238,"threads_started":1}
I20260923 20:50:11.285928 20908 raft_consensus.cc:482] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:50:11.286137 20908 raft_consensus.cc:504] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c2072da61e4ae9a0c73a1811d17023" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 41749 } } peers { permanent_uuid: "db2f2ff1dd7949128c285a8c98295ae3" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 40607 } } peers { permanent_uuid: "6ef2a69fc4bb442b8f7fe429b3def5c4" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 40927 } }
I20260923 20:50:11.286927 20908 leader_election.cc:288] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749), db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607)
I20260923 20:50:11.287909 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:11.286795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:53461)
0923 20:50:11.287473 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:11.287581 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":551,"threads_started":1}
W20260923 20:50:11.287953 20481 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1c2072da61e4ae9a0c73a1811d17023 (127.5.71.131:41749): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
W20260923 20:50:11.288152 20480 leader_election.cc:341] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db2f2ff1dd7949128c285a8c98295ae3 (127.5.71.130:40607): Not found: Tablet not found: df171e0f50e54bc29c89c3c94198e028
I20260923 20:50:11.288328 20480 leader_election.cc:302] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ef2a69fc4bb442b8f7fe429b3def5c4; no voters: db2f2ff1dd7949128c285a8c98295ae3, f1c2072da61e4ae9a0c73a1811d17023
I20260923 20:50:11.288615 20908 raft_consensus.cc:2726] T df171e0f50e54bc29c89c3c94198e028 P 6ef2a69fc4bb442b8f7fe429b3def5c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:50:11.317467 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:11.316510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.129:48803)
0923 20:50:11.317093 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:11.317245 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":288,"threads_started":1}
I20260923 20:50:11.795547 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:11.794749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.130:34891)
0923 20:50:11.795190 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:11.795350 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":219,"threads_started":1}
I20260923 20:50:12.259965 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 20429
I20260923 20:50:12.283982 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:12.261335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.5.71.129:40927)
0923 20:50:12.263314 (+ 1979us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:12.263322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:12.263350 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:50:12.263448 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:12.263467 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:12.263700 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:12.263917 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:12.265425 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:12.266419 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:12.267857 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:12.268451 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:12.268604 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:12.277187 (+ 8583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:12.277205 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:12.277224 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:12.277272 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:12.280810 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:12.281654 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:12.281662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:12.281669 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:12.281810 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:12.282233 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:12.282240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:12.282243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:12.282622 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:12.282953 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:12.283306 (+ 353us) server_negotiation.cc:299] Negotiation successful
0923 20:50:12.283605 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1790,"thread_start_us":1667,"threads_started":1}
I20260923 20:50:12.285473 20588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:12.289994 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:12.289145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45817 (local address 127.5.71.131:52369)
0923 20:50:12.289661 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:12.289788 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":281,"threads_started":1}
I20260923 20:50:12.382557 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 20429
I20260923 20:50:12.399530 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 20597
I20260923 20:50:12.415671 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:12.400408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.5.71.130:40607)
0923 20:50:12.400939 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:12.400943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:12.400959 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:50:12.401012 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:12.401015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:12.401071 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:12.401197 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:12.402699 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:12.403516 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:12.405080 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:12.405548 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:12.405702 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:12.408816 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:12.408823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:12.408826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:12.408869 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:12.412399 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:12.413328 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:12.413340 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:12.413343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:12.413494 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:12.413940 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:12.413947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:12.413951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:12.414274 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:12.414625 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:12.414943 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 20:50:12.415223 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":275,"threads_started":1}
I20260923 20:50:12.416679 20720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:12.508823 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 20597
I20260923 20:50:12.530068 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 20728
I20260923 20:50:12.545646 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:12.531012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.5.71.131:41749)
0923 20:50:12.531636 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:12.531641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:12.531658 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:50:12.531718 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:12.531721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:12.531853 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:12.531978 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:12.533464 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:12.534313 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:12.535567 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:12.536085 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:12.536242 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:12.539360 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:12.539366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:12.539369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:12.539411 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:12.542659 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:12.543464 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:12.543471 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:12.543474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:12.543641 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:12.544096 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:12.544101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:12.544104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:12.544413 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:12.544708 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:12.545011 (+ 303us) server_negotiation.cc:299] Negotiation successful
0923 20:50:12.545251 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":314,"threads_started":1}
I20260923 20:50:12.546562 20851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:12.637683 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 20728
2026-09-23T20:50:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25306 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:50:12 dist-test-slave-dn29 krb5kdc[20922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:50:12 dist-test-slave-dn29 krb5kdc[20922](info): set up 2 sockets
Sep 23 20:50:12 dist-test-slave-dn29 krb5kdc[20922](info): commencing operation
krb5kdc: starting...
W20260923 20:50:13.829388 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:50:14 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:50:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:50:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:50:14.113032 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45223
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:50:14.473400 20935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:14.473742 20935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:14.473901 20935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:14.481873 20935 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 20:50:14.481995 20935 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 20:50:14.482038 20935 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 20:50:14.482131 20935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:50:14.482215 20935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:14.482321 20935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:14.482374 20935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:50:14.482471 20935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:50:14.494028 20935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:14.495831 20935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:14.497371 20935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:14.511209 20940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:14.511119 20941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:14.511633 20935 server_base.cc:1034] running on GCE node
W20260923 20:50:14.516235 20943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:14.518046 20935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:14.520992 20935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:14.522373 20935 hybrid_clock.cc:648] HybridClock initialized: now 1790196614522233 us; error 112 us; skew 500 ppm
I20260923 20:50:14.522887 20935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:14 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196614, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:14.528445 20935 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:50:14.530786 20935 webserver.cc:460] Webserver started at http://127.5.71.190:43961/ using document root <none> and password file <none>
I20260923 20:50:14.531946 20935 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:14.532099 20935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:14.532582 20935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:50:14.537549 20935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "1d272365f98c4385ad88884b7b4f973a"
format_stamp: "Formatted at 2026-09-23 20:50:14 on dist-test-slave-dn29"
server_key: "9d1d43f9a036b7d86088900ddae582ce"
server_key_iv: "19c231d3db3d2b8868917c0fdece368b"
server_key_version: "encryptionkey@0"
I20260923 20:50:14.538761 20935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "1d272365f98c4385ad88884b7b4f973a"
format_stamp: "Formatted at 2026-09-23 20:50:14 on dist-test-slave-dn29"
server_key: "9d1d43f9a036b7d86088900ddae582ce"
server_key_iv: "19c231d3db3d2b8868917c0fdece368b"
server_key_version: "encryptionkey@0"
I20260923 20:50:14.546381 20935 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260923 20:50:14.550217 20950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:14.551755 20935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:50:14.552003 20935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "1d272365f98c4385ad88884b7b4f973a"
format_stamp: "Formatted at 2026-09-23 20:50:14 on dist-test-slave-dn29"
server_key: "9d1d43f9a036b7d86088900ddae582ce"
server_key_iv: "19c231d3db3d2b8868917c0fdece368b"
server_key_version: "encryptionkey@0"
I20260923 20:50:14.552212 20935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:14.568233 20935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:14.573226 20935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:14.573474 20935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:14.596438 20935 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45223
I20260923 20:50:14.596453 21001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45223 every 8 connection(s)
I20260923 20:50:14.597942 20935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:50:14.600603 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 20935
I20260923 20:50:14.600999 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:50:14.601655 5406 external_mini_cluster.cc:1428] Setting key b73769d38a1c9df24aa2ba27f0cfa8e4
I20260923 20:50:14.602615 21002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:50:14 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:14.616729 21002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap starting.
I20260923 20:50:14.620424 21002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:14.621608 21002 log.cc:824] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:14.624881 21002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: No bootstrap required, opened a new log
I20260923 20:50:14.625495 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:14.603878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46604 (local address 127.5.71.190:45223)
0923 20:50:14.604724 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:14.604735 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:14.604775 (+ 40us) server_negotiation.cc:407] Connection header received
0923 20:50:14.606422 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:14.606445 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:14.606861 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:14.607315 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:14.608926 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:14.610306 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:14.611781 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:14.612551 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:14.612786 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:14.617320 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:14.617329 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:14.617347 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:14.617390 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:14.620907 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:14.621727 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:14.621734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:14.621742 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:14.621970 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:14.622401 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:14.622406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:14.622409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:14.623029 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:14.623345 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:14.623907 (+ 562us) server_negotiation.cc:299] Negotiation successful
0923 20:50:14.624353 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":384,"threads_started":1}
I20260923 20:50:14.633585 21002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:14.634037 21002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:14.634166 21002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d272365f98c4385ad88884b7b4f973a, State: Initialized, Role: FOLLOWER
I20260923 20:50:14.634824 21002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:14.635128 21002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:50:14.635326 21002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:50:14.635515 21002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:14.637812 21002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:14.638296 21002 leader_election.cc:302] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d272365f98c4385ad88884b7b4f973a; no voters:
I20260923 20:50:14.638873 21002 leader_election.cc:288] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:50:14.639057 21007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:50:14.639909 21007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 LEADER]: Becoming Leader. State: Replica: 1d272365f98c4385ad88884b7b4f973a, State: Running, Role: LEADER
I20260923 20:50:14.640561 21007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:14.641247 21002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:50:14.643409 21009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d272365f98c4385ad88884b7b4f973a. Latest consensus state: current_term: 1 leader_uuid: "1d272365f98c4385ad88884b7b4f973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } } }
I20260923 20:50:14.644028 21009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: This master's current role is: LEADER
I20260923 20:50:14.643975 21008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d272365f98c4385ad88884b7b4f973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } } }
I20260923 20:50:14.644304 21008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: This master's current role is: LEADER
I20260923 20:50:14.649324 21015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:50:14.653951 21015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:50:14.664155 21015 catalog_manager.cc:1347] Generated new cluster ID: b0e56be013684cc4affc3c3bbffc6b71
I20260923 20:50:14.664266 21015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:50:14.672624 21015 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:50:14.674118 21015 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:50:14.680939 21015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Generated new TSK 0
I20260923 20:50:14.681896 21015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:50:14.752877 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45223
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:50:15.114730 21028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:15.115134 21028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:15.115322 21028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:15.123342 21028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:15.123551 21028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:15.123755 21028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:50:15.134657 21028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:15.136328 21028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:15.137856 21028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:15.153849 21034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:15.153858 21036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:15.153898 21033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:15.156325 21028 server_base.cc:1034] running on GCE node
I20260923 20:50:15.157051 21028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:15.158152 21028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:15.159382 21028 hybrid_clock.cc:648] HybridClock initialized: now 1790196615159326 us; error 33 us; skew 500 ppm
I20260923 20:50:15.159686 21028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:15 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196615, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:15.163249 21028 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:50:15.164889 21028 webserver.cc:460] Webserver started at http://127.5.71.129:43111/ using document root <none> and password file <none>
I20260923 20:50:15.165616 21028 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:15.165779 21028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:15.166102 21028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:50:15.169252 21028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "e44dec8a18b6439da47ceb3945af781a"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "c742835eec13b19572543a214f4a4ee2"
server_key_iv: "f74b961c0f9477ca93be0330b5c53019"
server_key_version: "encryptionkey@0"
I20260923 20:50:15.170038 21028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "e44dec8a18b6439da47ceb3945af781a"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "c742835eec13b19572543a214f4a4ee2"
server_key_iv: "f74b961c0f9477ca93be0330b5c53019"
server_key_version: "encryptionkey@0"
I20260923 20:50:15.175469 21028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:50:15.179394 21043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:15.181625 21028 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260923 20:50:15.181864 21028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "e44dec8a18b6439da47ceb3945af781a"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "c742835eec13b19572543a214f4a4ee2"
server_key_iv: "f74b961c0f9477ca93be0330b5c53019"
server_key_version: "encryptionkey@0"
I20260923 20:50:15.182116 21028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:15.199194 21028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:15.204088 21028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:15.204413 21028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:15.205472 21028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:50:15.206959 21028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:50:15.207068 21028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:15.207218 21028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:50:15.207296 21028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:15.243239 21028 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:35427
I20260923 20:50:15.243351 21155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:35427 every 8 connection(s)
I20260923 20:50:15.244711 21028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:50:15.252668 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21028
I20260923 20:50:15.252939 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:50:15.253799 5406 external_mini_cluster.cc:1428] Setting key ed68a974c6399bbf587e100b656064c8
Sep 23 20:50:15 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196615, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:15.270843 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:15.247757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:57831 (local address 127.5.71.190:45223)
0923 20:50:15.248275 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:15.248282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:15.249482 (+ 1200us) server_negotiation.cc:407] Connection header received
0923 20:50:15.251567 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:15.251572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:15.251653 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:15.251980 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:15.254648 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.255524 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.258235 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.258673 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.258808 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:15.263933 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:15.263939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:15.263943 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:15.263981 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:15.267244 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:15.268338 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:15.268343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:15.268345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:15.268477 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:15.268968 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:15.268972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:15.268974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:15.269275 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:15.269444 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:15.270212 (+ 768us) server_negotiation.cc:299] Negotiation successful
0923 20:50:15.270465 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":240,"threads_started":1}
I20260923 20:50:15.271979 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:15.248488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:57831)
0923 20:50:15.249298 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:15.249338 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:15.251206 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:15.252330 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:15.252345 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:15.253242 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:50:15.254283 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:15.254311 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.255821 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.255826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:15.258037 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:15.258059 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.258969 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.258980 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:15.259470 (+ 490us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:15.260328 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:50:15.260364 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:50:15.263565 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:50:15.267472 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:15.267481 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:15.267498 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:15.268165 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:15.268605 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:15.268610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:15.268615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:15.268828 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:15.269414 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:50:15.269423 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:50:15.269813 (+ 390us) client_negotiation.cc:769] Sending connection context
0923 20:50:15.270101 (+ 288us) client_negotiation.cc:240] Negotiation successful
0923 20:50:15.270688 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":321,"threads_started":1}
I20260923 20:50:15.273906 21156 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45223
I20260923 20:50:15.274279 21156 heartbeater.cc:464] Registering TS with master...
I20260923 20:50:15.275130 21156 heartbeater.cc:511] Master 127.5.71.190:45223 requested a full tablet report, sending...
I20260923 20:50:15.278043 20967 ts_manager.cc:194] Registered new tserver with Master: e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427)
I20260923 20:50:15.280558 20967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:57831
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:50:15.321015 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45223
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:50:15.681887 21162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:15.682206 21162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:15.682318 21162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:15.690706 21162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:15.690847 21162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:15.690985 21162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:50:15.701489 21162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:15.702857 21162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:15.704262 21162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:15.720382 21168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:15.720404 21167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:15.720476 21170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:15.722079 21162 server_base.cc:1034] running on GCE node
I20260923 20:50:15.722831 21162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:15.724017 21162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:15.725309 21162 hybrid_clock.cc:648] HybridClock initialized: now 1790196615725241 us; error 53 us; skew 500 ppm
I20260923 20:50:15.725718 21162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:15 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196615, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:15.729542 21162 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:50:15.731191 21162 webserver.cc:460] Webserver started at http://127.5.71.130:39869/ using document root <none> and password file <none>
I20260923 20:50:15.731976 21162 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:15.732086 21162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:15.732369 21162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:50:15.735388 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "bcd4e89be4924cd58df6745904779990"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "5694eb14b8ba0765624dda7372a7c1b7"
server_key_iv: "6e3686223e520879c46a4100b484a998"
server_key_version: "encryptionkey@0"
I20260923 20:50:15.736212 21162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "bcd4e89be4924cd58df6745904779990"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "5694eb14b8ba0765624dda7372a7c1b7"
server_key_iv: "6e3686223e520879c46a4100b484a998"
server_key_version: "encryptionkey@0"
I20260923 20:50:15.742311 21162 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:50:15.746430 21177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:15.748112 21162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:50:15.748327 21162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "bcd4e89be4924cd58df6745904779990"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "5694eb14b8ba0765624dda7372a7c1b7"
server_key_iv: "6e3686223e520879c46a4100b484a998"
server_key_version: "encryptionkey@0"
I20260923 20:50:15.748559 21162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:15.766170 21162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:15.770604 21162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:15.770924 21162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:15.772110 21162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:50:15.773521 21162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:50:15.773627 21162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:15.773782 21162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:50:15.773860 21162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:15.811556 21162 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:35549
I20260923 20:50:15.811676 21289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:35549 every 8 connection(s)
I20260923 20:50:15.813004 21162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:50:15.818426 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21162
I20260923 20:50:15.818744 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:50:15.819319 5406 external_mini_cluster.cc:1428] Setting key 7cbec13e92902d4f4867f059588deb9d
Sep 23 20:50:15 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196615, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:15.836233 21292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:15.816034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:39409 (local address 127.5.71.190:45223)
0923 20:50:15.816561 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:15.816567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:15.817526 (+ 959us) server_negotiation.cc:407] Connection header received
0923 20:50:15.819736 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:15.819742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:15.819868 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:15.819994 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:15.821647 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.822402 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.824107 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.824544 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.824688 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:15.828851 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:15.828862 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:15.828881 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:15.828935 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:15.832295 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:15.833406 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:15.833411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:15.833413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:15.833575 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:15.834192 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:15.834197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:15.834199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:15.834509 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:15.834640 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:15.835637 (+ 997us) server_negotiation.cc:299] Negotiation successful
0923 20:50:15.835926 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":293,"threads_started":1}
I20260923 20:50:15.837843 21293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:15.816549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:39409)
0923 20:50:15.817337 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:15.817377 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:15.819387 (+ 2010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:15.820266 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:15.820276 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:15.820849 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:50:15.821450 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:15.821468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.822581 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.822585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:15.823954 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:15.823966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:15.824811 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:15.824819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:15.825065 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:15.825815 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:50:15.825841 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:50:15.828508 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:50:15.832475 (+ 3967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:15.832490 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:15.832514 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:15.833211 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:15.833713 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:15.833720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:15.833726 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:15.833989 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:15.834651 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:50:15.834662 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:50:15.835172 (+ 510us) client_negotiation.cc:769] Sending connection context
0923 20:50:15.835491 (+ 319us) client_negotiation.cc:240] Negotiation successful
0923 20:50:15.836145 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":343,"threads_started":1}
I20260923 20:50:15.839696 21290 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45223
I20260923 20:50:15.840088 21290 heartbeater.cc:464] Registering TS with master...
I20260923 20:50:15.840979 21290 heartbeater.cc:511] Master 127.5.71.190:45223 requested a full tablet report, sending...
I20260923 20:50:15.842878 20967 ts_manager.cc:194] Registered new tserver with Master: bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549)
I20260923 20:50:15.843743 20967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:39409
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:50:15.881279 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45223
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:50:16.245664 21296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:16.245950 21296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:16.246122 21296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:16.253808 21296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:16.253929 21296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:16.254045 21296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:50:16.264545 21296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:16.265952 21296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:16.267259 21296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:16.281663 21301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:16.281796 21304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:16.281626 21302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:16.282618 21296 server_base.cc:1034] running on GCE node
I20260923 20:50:16.283427 21296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:16.284614 21296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:16.284709 21156 heartbeater.cc:503] Master 127.5.71.190:45223 was elected leader, sending a full tablet report...
I20260923 20:50:16.285957 21296 hybrid_clock.cc:648] HybridClock initialized: now 1790196616285822 us; error 121 us; skew 500 ppm
I20260923 20:50:16.286410 21296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:16 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196616, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:16.290201 21296 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:50:16.292330 21296 webserver.cc:460] Webserver started at http://127.5.71.131:33695/ using document root <none> and password file <none>
I20260923 20:50:16.293164 21296 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:16.293325 21296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:16.293689 21296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:50:16.297148 21296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5"
format_stamp: "Formatted at 2026-09-23 20:50:16 on dist-test-slave-dn29"
server_key: "55afd6600e135c1189cc4b9d5a14c9d3"
server_key_iv: "596da04367d42420141b12f9fc49646f"
server_key_version: "encryptionkey@0"
I20260923 20:50:16.298023 21296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5"
format_stamp: "Formatted at 2026-09-23 20:50:16 on dist-test-slave-dn29"
server_key: "55afd6600e135c1189cc4b9d5a14c9d3"
server_key_iv: "596da04367d42420141b12f9fc49646f"
server_key_version: "encryptionkey@0"
I20260923 20:50:16.304502 21296 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260923 20:50:16.308691 21311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:16.310276 21296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260923 20:50:16.310493 21296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5"
format_stamp: "Formatted at 2026-09-23 20:50:16 on dist-test-slave-dn29"
server_key: "55afd6600e135c1189cc4b9d5a14c9d3"
server_key_iv: "596da04367d42420141b12f9fc49646f"
server_key_version: "encryptionkey@0"
I20260923 20:50:16.310837 21296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:16.337348 21296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:16.342839 21296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:16.343175 21296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:16.344316 21296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:50:16.345889 21296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:50:16.346004 21296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:16.346154 21296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:50:16.346232 21296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:16.383697 21296 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:35435
I20260923 20:50:16.383734 21423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:35435 every 8 connection(s)
I20260923 20:50:16.385068 21296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:50:16.390322 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21296
I20260923 20:50:16.390641 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:50:16.391191 5406 external_mini_cluster.cc:1428] Setting key 7f85fc4a2439763ba3e661b7703ee3f9
Sep 23 20:50:16 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196616, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:16.408067 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.388303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:36055 (local address 127.5.71.190:45223)
0923 20:50:16.388780 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.388785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.390077 (+ 1292us) server_negotiation.cc:407] Connection header received
0923 20:50:16.392365 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.392370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.392470 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.392601 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:16.394213 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.395051 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.396848 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.397277 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.397425 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.401257 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:16.401267 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:16.401271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:16.401318 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:16.404480 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:16.405363 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:16.405368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:16.405370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:16.405515 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:16.406062 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:16.406067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:16.406070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:16.406377 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:16.406564 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.407332 (+ 768us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.407633 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":231,"threads_started":1}
I20260923 20:50:16.409196 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.389032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:36055)
0923 20:50:16.389907 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:16.389947 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:16.392046 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:16.392870 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:16.392878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:16.393422 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:50:16.394032 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.394051 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.395234 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.395241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.396659 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.396679 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.397528 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.397535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.397787 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.398531 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:50:16.398556 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:50:16.401011 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:50:16.404657 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:16.404668 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:16.404690 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:16.405206 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:16.405690 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:16.405695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:16.405699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:16.405922 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:16.406517 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:50:16.406526 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:50:16.406918 (+ 392us) client_negotiation.cc:769] Sending connection context
0923 20:50:16.407222 (+ 304us) client_negotiation.cc:240] Negotiation successful
0923 20:50:16.407908 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":342,"threads_started":1}
I20260923 20:50:16.411032 21424 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45223
I20260923 20:50:16.411370 21424 heartbeater.cc:464] Registering TS with master...
I20260923 20:50:16.412262 21424 heartbeater.cc:511] Master 127.5.71.190:45223 requested a full tablet report, sending...
I20260923 20:50:16.414141 20967 ts_manager.cc:194] Registered new tserver with Master: 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
I20260923 20:50:16.415246 20967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:36055
I20260923 20:50:16.420226 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:50:16.440814 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.425177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.5.71.190:45223)
0923 20:50:16.425504 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.425514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.425941 (+ 427us) server_negotiation.cc:407] Connection header received
0923 20:50:16.426187 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.426191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.426266 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.426387 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:16.428280 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.429104 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.430743 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.431210 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.431427 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.434343 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:16.434348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:16.434351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:16.434390 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:16.437570 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:16.438490 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:16.438496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:16.438500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:16.438660 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:16.439090 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:16.439098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:16.439104 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:16.439482 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:16.439836 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.440178 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.440453 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260923 20:50:16.446354 20967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:50:16.454492 20967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:50:16.495724 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.470435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.5.71.130:35549)
0923 20:50:16.485941 (+ 15506us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.485950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.485984 (+ 34us) server_negotiation.cc:407] Connection header received
0923 20:50:16.486108 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.486120 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.486429 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.486836 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:16.487848 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.489452 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.492928 (+ 3476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.494153 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.494349 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.494613 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.495158 (+ 545us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.495364 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15315,"thread_start_us":15127,"threads_started":1}
I20260923 20:50:16.495882 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.469660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:35435 (local address 127.0.0.1:43330)
0923 20:50:16.483078 (+ 13418us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:16.483142 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:16.483435 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:16.487388 (+ 3953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:16.487393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:16.487429 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:16.488157 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.488170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.490232 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.490240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.493014 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.493028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.494913 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.494922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.495115 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.495150 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 20:50:16.495232 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 20:50:16.495380 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13021,"mutex_wait_us":128,"thread_start_us":15029,"threads_started":1}
I20260923 20:50:16.495826 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.469742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:35549 (local address 127.0.0.1:58520)
0923 20:50:16.484448 (+ 14706us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:16.484469 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:16.484599 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:16.486886 (+ 2287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:16.486915 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:16.486958 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:16.487537 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.487555 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.489627 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.489636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.492678 (+ 3042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.492700 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.494402 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.494409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.494648 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.494717 (+ 69us) client_negotiation.cc:769] Sending connection context
0923 20:50:16.494929 (+ 212us) client_negotiation.cc:240] Negotiation successful
0923 20:50:16.495185 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14572,"mutex_wait_us":96,"thread_start_us":12760,"threads_started":1}
I20260923 20:50:16.496662 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.469937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.5.71.131:35435)
0923 20:50:16.486565 (+ 16628us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.486574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.486611 (+ 37us) server_negotiation.cc:407] Connection header received
0923 20:50:16.486711 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.486718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.487161 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.487399 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:16.488402 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.490063 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.493201 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.494655 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.494817 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.495016 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.496105 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.496302 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16432,"thread_start_us":16290,"threads_started":1}
I20260923 20:50:16.497179 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.471071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.5.71.129:35427)
0923 20:50:16.485424 (+ 14353us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.485431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.486359 (+ 928us) server_negotiation.cc:407] Connection header received
0923 20:50:16.486463 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.486474 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.486890 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.488039 (+ 1149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:16.489022 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.490499 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.492990 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.494544 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.494699 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.494977 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.496558 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.496766 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14175,"thread_start_us":14048,"threads_started":1}
I20260923 20:50:16.497648 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.470150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:33708)
0923 20:50:16.486069 (+ 15919us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:16.486088 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:16.486236 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:16.488359 (+ 2123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:16.488364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:16.488393 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:16.488826 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.488839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.490655 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.490663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.492796 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.492811 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.495149 (+ 2338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.495157 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.495451 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.495507 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 20:50:16.495606 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 20:50:16.495764 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15769,"thread_start_us":15599,"threads_started":1}
I20260923 20:50:16.500982 21225 tablet_service.cc:1467] Processing CreateTablet for tablet 348c0adb5ffc433f8fea52aa954ac71d (DEFAULT_TABLE table=test-table [id=761329b6fde34381b5008d3159267d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:50:16.500982 21359 tablet_service.cc:1467] Processing CreateTablet for tablet 348c0adb5ffc433f8fea52aa954ac71d (DEFAULT_TABLE table=test-table [id=761329b6fde34381b5008d3159267d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:50:16.501264 21091 tablet_service.cc:1467] Processing CreateTablet for tablet 348c0adb5ffc433f8fea52aa954ac71d (DEFAULT_TABLE table=test-table [id=761329b6fde34381b5008d3159267d02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:50:16.502635 21225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348c0adb5ffc433f8fea52aa954ac71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:50:16.502633 21359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348c0adb5ffc433f8fea52aa954ac71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:50:16.502650 21091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348c0adb5ffc433f8fea52aa954ac71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:50:16.520292 21444 tablet_bootstrap.cc:492] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Bootstrap starting.
I20260923 20:50:16.524287 21445 tablet_bootstrap.cc:492] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Bootstrap starting.
I20260923 20:50:16.524350 21444 tablet_bootstrap.cc:654] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:16.524618 21443 tablet_bootstrap.cc:492] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Bootstrap starting.
I20260923 20:50:16.527349 21444 log.cc:824] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:16.528424 21445 tablet_bootstrap.cc:654] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:16.528476 21443 tablet_bootstrap.cc:654] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:16.529624 21443 log.cc:824] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:16.529716 21445 log.cc:824] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:16.531353 21444 tablet_bootstrap.cc:492] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: No bootstrap required, opened a new log
I20260923 20:50:16.531682 21444 ts_tablet_manager.cc:1397] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260923 20:50:16.532706 21443 tablet_bootstrap.cc:492] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: No bootstrap required, opened a new log
I20260923 20:50:16.532998 21443 ts_tablet_manager.cc:1397] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260923 20:50:16.533609 21445 tablet_bootstrap.cc:492] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: No bootstrap required, opened a new log
I20260923 20:50:16.533869 21445 ts_tablet_manager.cc:1397] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260923 20:50:16.541028 21443 raft_consensus.cc:348] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.541579 21443 raft_consensus.cc:374] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:16.541692 21443 raft_consensus.cc:729] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd4e89be4924cd58df6745904779990, State: Initialized, Role: FOLLOWER
I20260923 20:50:16.542438 21443 consensus_queue.cc:260] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.543210 21444 raft_consensus.cc:348] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.543700 21444 raft_consensus.cc:374] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:16.543815 21444 raft_consensus.cc:729] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44dec8a18b6439da47ceb3945af781a, State: Initialized, Role: FOLLOWER
I20260923 20:50:16.543769 21290 heartbeater.cc:503] Master 127.5.71.190:45223 was elected leader, sending a full tablet report...
I20260923 20:50:16.544538 21443 ts_tablet_manager.cc:1428] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260923 20:50:16.544548 21444 consensus_queue.cc:260] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.545113 21445 raft_consensus.cc:348] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.546001 21445 raft_consensus.cc:374] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:16.546106 21445 raft_consensus.cc:729] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, State: Initialized, Role: FOLLOWER
I20260923 20:50:16.546516 21444 ts_tablet_manager.cc:1428] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260923 20:50:16.546893 21445 consensus_queue.cc:260] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.548801 21424 heartbeater.cc:503] Master 127.5.71.190:45223 was elected leader, sending a full tablet report...
I20260923 20:50:16.550062 21445 ts_tablet_manager.cc:1428] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Time spent starting tablet: real 0.016s user 0.010s sys 0.007s
W20260923 20:50:16.565275 21291 tablet.cc:2369] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:50:16.636983 21425 tablet.cc:2369] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:50:16.748382 21157 tablet.cc:2369] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:50:16.907475 21451 raft_consensus.cc:482] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:50:16.907859 21451 raft_consensus.cc:504] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.910867 21451 leader_election.cc:288] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427)
I20260923 20:50:16.919083 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.910374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:59279 (local address 127.5.71.129:35427)
0923 20:50:16.910600 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.910607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.910912 (+ 305us) server_negotiation.cc:407] Connection header received
0923 20:50:16.911315 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.911319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.911384 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.911518 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:16.912095 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.913163 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.916207 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.917812 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.917978 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.918050 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.918669 (+ 619us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.918795 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 20:50:16.919281 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.910137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.5.71.131:59279)
0923 20:50:16.910777 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:16.910799 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:16.911036 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:16.911564 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:16.911571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:16.911602 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:16.911921 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.911932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.913326 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.913331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.915959 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.915980 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.918017 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.918026 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.918262 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.918404 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 20:50:16.918518 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 20:50:16.918789 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":158,"thread_start_us":470,"threads_started":1}
I20260923 20:50:16.919420 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.910392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:44515 (local address 127.5.71.130:35549)
0923 20:50:16.910603 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.910610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.911480 (+ 870us) server_negotiation.cc:407] Connection header received
0923 20:50:16.911665 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.911673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.911746 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.911881 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:16.912440 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.913453 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.916206 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.917738 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.917965 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.918050 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.918979 (+ 929us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.919154 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 20:50:16.919577 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.910521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:35549 (local address 127.5.71.131:44515)
0923 20:50:16.911354 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:16.911369 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:16.911517 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:16.911909 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:16.911914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:16.911937 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:16.912281 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.912290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.913596 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.913601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.915962 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:16.915979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.918055 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.918060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:16.918267 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.918406 (+ 139us) client_negotiation.cc:769] Sending connection context
0923 20:50:16.918907 (+ 501us) client_negotiation.cc:240] Negotiation successful
0923 20:50:16.919065 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":99,"thread_start_us":390,"threads_started":1}
I20260923 20:50:16.920819 21245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348c0adb5ffc433f8fea52aa954ac71d" candidate_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e89be4924cd58df6745904779990" is_pre_election: true
I20260923 20:50:16.921438 21245 raft_consensus.cc:2454] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5a5e3f6cd64ceca18e4ace46f3b7e5 in term 0.
I20260923 20:50:16.921617 21111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348c0adb5ffc433f8fea52aa954ac71d" candidate_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44dec8a18b6439da47ceb3945af781a" is_pre_election: true
I20260923 20:50:16.922166 21111 raft_consensus.cc:2454] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e5a5e3f6cd64ceca18e4ace46f3b7e5 in term 0.
I20260923 20:50:16.922271 21314 leader_election.cc:302] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990; no voters:
I20260923 20:50:16.922780 21451 raft_consensus.cc:2781] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:50:16.922940 21451 raft_consensus.cc:482] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:50:16.923074 21451 raft_consensus.cc:3037] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:16.925400 21451 raft_consensus.cc:504] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.926035 21451 leader_election.cc:288] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [CANDIDATE]: Term 1 election: Requested vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427)
I20260923 20:50:16.926601 21111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348c0adb5ffc433f8fea52aa954ac71d" candidate_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44dec8a18b6439da47ceb3945af781a"
I20260923 20:50:16.926641 21245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348c0adb5ffc433f8fea52aa954ac71d" candidate_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e89be4924cd58df6745904779990"
I20260923 20:50:16.926849 21245 raft_consensus.cc:3037] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:16.927166 21111 raft_consensus.cc:3037] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:16.929246 21245 raft_consensus.cc:2454] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5a5e3f6cd64ceca18e4ace46f3b7e5 in term 1.
I20260923 20:50:16.929519 21111 raft_consensus.cc:2454] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e5a5e3f6cd64ceca18e4ace46f3b7e5 in term 1.
I20260923 20:50:16.929718 21314 leader_election.cc:302] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990; no voters:
I20260923 20:50:16.930114 21451 raft_consensus.cc:2781] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:50:16.930675 21451 raft_consensus.cc:686] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 LEADER]: Becoming Leader. State: Replica: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, State: Running, Role: LEADER
I20260923 20:50:16.931185 21451 consensus_queue.cc:237] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } }
I20260923 20:50:16.936195 20965 catalog_manager.cc:5495] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } health_report { overall_health: UNKNOWN } } }
I20260923 20:50:16.972694 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:16.965629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.5.71.131:35435)
0923 20:50:16.965871 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:16.965876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:16.966472 (+ 596us) server_negotiation.cc:407] Connection header received
0923 20:50:16.966605 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:16.966609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:16.966692 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:16.966907 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:50:16.967420 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.968430 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.970126 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:16.970589 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:16.970722 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:16.971315 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:50:16.972028 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:50:16.972162 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:16.972380 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 20:50:16.972464 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260923 20:50:16.982275 21111 raft_consensus.cc:1261] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Refusing update from remote peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:50:16.982295 21245 raft_consensus.cc:1261] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Refusing update from remote peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:50:16.983387 21454 consensus_queue.cc:1035] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:50:16.983865 21451 consensus_queue.cc:1035] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:50:16.996760 21458 mvcc.cc:205] Tried to move back new op lower bound from 7332645343148040192 to 7332645342954283008. Current Snapshot: MvccSnapshot[applied={T|T < 7332645343148040192}]
I20260923 20:50:17.013839 20965 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:50:17.014179 20965 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:50:17.018658 20965 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Sending DeleteTablet for 3 replicas of tablet 348c0adb5ffc433f8fea52aa954ac71d
I20260923 20:50:17.020401 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21028
I20260923 20:50:17.020304 21225 tablet_service.cc:1514] Processing DeleteTablet for tablet 348c0adb5ffc433f8fea52aa954ac71d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:50:17 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:58520
I20260923 20:50:17.020481 21091 tablet_service.cc:1514] Processing DeleteTablet for tablet 348c0adb5ffc433f8fea52aa954ac71d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:50:17 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:33708
I20260923 20:50:17.021596 21359 tablet_service.cc:1514] Processing DeleteTablet for tablet 348c0adb5ffc433f8fea52aa954ac71d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:50:17 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:43330
I20260923 20:50:17.030102 21471 tablet_replica.cc:330] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: stopping tablet replica
I20260923 20:50:17.032761 21471 raft_consensus.cc:2229] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:50:17.034375 21471 raft_consensus.cc:2258] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:50:17.038065 21469 tablet_replica.cc:330] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: stopping tablet replica
I20260923 20:50:17.038730 21471 ts_tablet_manager.cc:1905] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:50:17.038734 21469 raft_consensus.cc:2229] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:50:17.039487 21469 raft_consensus.cc:2258] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:50:17.041798 21469 ts_tablet_manager.cc:1905] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 20:50:17 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:50:17.045518 21468 tablet_replica.cc:330] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: stopping tablet replica
I20260923 20:50:17.046168 21468 raft_consensus.cc:2229] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:50:17.047340 21471 ts_tablet_manager.cc:1918] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:50:17.047561 21471 log.cc:1196] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/348c0adb5ffc433f8fea52aa954ac71d
I20260923 20:50:17.048139 21468 raft_consensus.cc:2258] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:50:17.048357 21471 ts_tablet_manager.cc:1939] T 348c0adb5ffc433f8fea52aa954ac71d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Deleting consensus metadata
I20260923 20:50:17.050885 20951 catalog_manager.cc:4841] TS 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): tablet 348c0adb5ffc433f8fea52aa954ac71d (table test-table [id=761329b6fde34381b5008d3159267d02]) successfully deleted
I20260923 20:50:17.052881 21468 ts_tablet_manager.cc:1905] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:50:17.053953 21469 ts_tablet_manager.cc:1918] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:50:17.054141 21469 log.cc:1196] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/348c0adb5ffc433f8fea52aa954ac71d
I20260923 20:50:17.054697 21469 ts_tablet_manager.cc:1939] T 348c0adb5ffc433f8fea52aa954ac71d P e44dec8a18b6439da47ceb3945af781a: Deleting consensus metadata
I20260923 20:50:17.055922 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:17.021706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.5.71.129:35427)
0923 20:50:17.021970 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:17.021979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:17.032438 (+ 10459us) server_negotiation.cc:407] Connection header received
0923 20:50:17.033054 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:17.033061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:17.033163 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:17.033241 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:17.035760 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:17.037009 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:17.039221 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:17.040025 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:17.040144 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:17.046479 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:17.046501 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:17.046521 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:17.046573 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:17.051308 (+ 4735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:17.052593 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:17.052604 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:17.052608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:17.052810 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:17.053430 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:17.053436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:17.053439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:17.053984 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:17.054336 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:17.055069 (+ 733us) server_negotiation.cc:299] Negotiation successful
0923 20:50:17.055444 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260923 20:50:17.057003 20952 catalog_manager.cc:4841] TS e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427): tablet 348c0adb5ffc433f8fea52aa954ac71d (table test-table [id=761329b6fde34381b5008d3159267d02]) successfully deleted
I20260923 20:50:17.057850 21151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:17.063061 21468 ts_tablet_manager.cc:1918] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:50:17.063279 21468 log.cc:1196] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/348c0adb5ffc433f8fea52aa954ac71d
I20260923 20:50:17.063862 21468 ts_tablet_manager.cc:1939] T 348c0adb5ffc433f8fea52aa954ac71d P bcd4e89be4924cd58df6745904779990: Deleting consensus metadata
I20260923 20:50:17.066129 20953 catalog_manager.cc:4841] TS bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): tablet 348c0adb5ffc433f8fea52aa954ac71d (table test-table [id=761329b6fde34381b5008d3159267d02]) successfully deleted
I20260923 20:50:17.155244 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21028
I20260923 20:50:17.183667 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21162
Sep 23 20:50:17 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:50:17.203367 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:17.184559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.5.71.130:35549)
0923 20:50:17.184763 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:17.184768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:17.184786 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:50:17.184927 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:17.184931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:17.184997 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:17.185081 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:17.187033 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:17.187905 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:17.190110 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:17.190513 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:17.190613 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:17.196880 (+ 6267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:17.196900 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:17.196919 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:17.196964 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:17.200275 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:17.201199 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:17.201209 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:17.201212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:17.201348 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:17.201751 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:17.201754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:17.201756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:17.202134 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:17.202362 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:17.202836 (+ 474us) server_negotiation.cc:299] Negotiation successful
0923 20:50:17.203006 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 20:50:17.204180 21285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:17.296213 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21162
I20260923 20:50:17.320386 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21296
Sep 23 20:50:17 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:50:17.345970 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:17.321300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.5.71.131:35435)
0923 20:50:17.321652 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:17.321664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:17.321689 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:50:17.321778 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:17.321783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:17.321862 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:17.322020 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:17.323631 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:17.324652 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:17.326279 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:17.326820 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:17.327000 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:17.338196 (+ 11196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:17.338214 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:17.338236 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:17.338298 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:17.342165 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:17.343327 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:17.343337 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:17.343339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:17.343513 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:17.344044 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:17.344049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:17.344051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:17.344611 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:17.344924 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:17.345275 (+ 351us) server_negotiation.cc:299] Negotiation successful
0923 20:50:17.345595 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260923 20:50:17.346930 21419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:17.478790 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21296
I20260923 20:50:17.497090 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 20935
I20260923 20:50:17.498523 20997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:17.615382 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 20935
I20260923 20:50:37.641094 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45223
--webserver_interface=127.5.71.190
--webserver_port=43961
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:50:38.008610 21476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:38.008944 21476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:38.009060 21476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:38.016680 21476 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 20:50:38.016798 21476 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 20:50:38.016841 21476 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 20:50:38.016888 21476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:50:38.016984 21476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:38.017108 21476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:38.017180 21476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:50:38.017282 21476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:50:38.028365 21476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=43961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:38.029809 21476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:38.031255 21476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:38.058470 21483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:38.058490 21482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:38.058535 21476 server_base.cc:1034] running on GCE node
W20260923 20:50:38.058969 21485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:38.060238 21476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:38.062551 21476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:38.064003 21476 hybrid_clock.cc:648] HybridClock initialized: now 1790196638063907 us; error 49 us; skew 500 ppm
I20260923 20:50:38.064539 21476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:38 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:38.069418 21476 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:50:38.071565 21476 webserver.cc:460] Webserver started at http://127.5.71.190:43961/ using document root <none> and password file <none>
I20260923 20:50:38.072625 21476 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:38.072806 21476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:38.079916 21476 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.003s
I20260923 20:50:38.083616 21492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:38.085594 21476 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:50:38.085868 21476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "1d272365f98c4385ad88884b7b4f973a"
format_stamp: "Formatted at 2026-09-23 20:50:14 on dist-test-slave-dn29"
server_key: "9d1d43f9a036b7d86088900ddae582ce"
server_key_iv: "19c231d3db3d2b8868917c0fdece368b"
server_key_version: "encryptionkey@0"
I20260923 20:50:38.087096 21476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:38.104167 21476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:38.109275 21476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:38.109629 21476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:38.132225 21476 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45223
I20260923 20:50:38.132258 21543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45223 every 8 connection(s)
I20260923 20:50:38.133989 21476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:50:38.140342 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21476
I20260923 20:50:38.141446 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:35427
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:45223
--webserver_port=43111
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:50:38.143286 21544 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:50:38.149149 21544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap starting.
I20260923 20:50:38.171609 21544 log.cc:824] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:38.185122 21544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:50:38.185734 21544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap complete.
I20260923 20:50:38.196735 21544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:38.197692 21544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d272365f98c4385ad88884b7b4f973a, State: Initialized, Role: FOLLOWER
I20260923 20:50:38.198516 21544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:38.198800 21544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:50:38.198935 21544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:50:38.199136 21544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 1 FOLLOWER]: Advancing to term 2
I20260923 20:50:38.203509 21544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:38.204135 21544 leader_election.cc:302] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d272365f98c4385ad88884b7b4f973a; no voters:
I20260923 20:50:38.204866 21544 leader_election.cc:288] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 20:50:38.205019 21548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Leader election won for term 2
I20260923 20:50:38.206182 21548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 LEADER]: Becoming Leader. State: Replica: 1d272365f98c4385ad88884b7b4f973a, State: Running, Role: LEADER
I20260923 20:50:38.206892 21548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:50:38.207405 21544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:50:38.209661 21550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d272365f98c4385ad88884b7b4f973a. Latest consensus state: current_term: 2 leader_uuid: "1d272365f98c4385ad88884b7b4f973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } } }
I20260923 20:50:38.210117 21550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: This master's current role is: LEADER
I20260923 20:50:38.209815 21549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d272365f98c4385ad88884b7b4f973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } } }
I20260923 20:50:38.210314 21549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: This master's current role is: LEADER
I20260923 20:50:38.221570 21555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:50:38.230644 21555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:50:38.232241 21555 catalog_manager.cc:1259] Loaded cluster ID: b0e56be013684cc4affc3c3bbffc6b71
I20260923 20:50:38.232421 21555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:50:38.236685 21555 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:50:38.238369 21555 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Loaded TSK: 0
I20260923 20:50:38.273990 21555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Generated new TSK 1
I20260923 20:50:38.274435 21555 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 20:50:38.534964 21546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:38.535343 21546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:38.535516 21546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:38.543397 21546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:38.543578 21546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:38.543758 21546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:50:38.554507 21546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:35427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=43111
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:38.556056 21546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:38.557569 21546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:38.573486 21572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:38.573558 21571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:38.573518 21574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:38.574005 21546 server_base.cc:1034] running on GCE node
I20260923 20:50:38.575035 21546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:38.576517 21546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:38.577831 21546 hybrid_clock.cc:648] HybridClock initialized: now 1790196638577741 us; error 70 us; skew 500 ppm
I20260923 20:50:38.578285 21546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:38 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:38.590241 21546 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:50:38.592422 21546 webserver.cc:460] Webserver started at http://127.5.71.129:43111/ using document root <none> and password file <none>
I20260923 20:50:38.593287 21546 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:38.593451 21546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:38.599494 21546 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260923 20:50:38.602682 21581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:38.604307 21546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:50:38.604511 21546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "e44dec8a18b6439da47ceb3945af781a"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "c742835eec13b19572543a214f4a4ee2"
server_key_iv: "f74b961c0f9477ca93be0330b5c53019"
server_key_version: "encryptionkey@0"
I20260923 20:50:38.605506 21546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:38.622797 21546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:38.627517 21546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:38.627903 21546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:38.629062 21546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:50:38.630581 21546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:50:38.630671 21546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:38.630806 21546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:50:38.630890 21546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:38.669122 21546 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:35427
I20260923 20:50:38.669222 21693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:35427 every 8 connection(s)
I20260923 20:50:38.670682 21546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:50:38.674904 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21546
I20260923 20:50:38.675865 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:35549
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:45223
--webserver_port=39869
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:50:38 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196638, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:38.700059 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:38.674213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:50561)
0923 20:50:38.675222 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:38.675281 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:38.678056 (+ 2775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:38.679776 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:38.679819 (+ 43us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:38.680592 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:50:38.681198 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:38.681218 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:38.683089 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:38.683102 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:38.685150 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:38.685169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:38.686271 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:38.686282 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:38.686634 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:38.687485 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:50:38.687523 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:50:38.690780 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:50:38.694968 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:38.694979 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:38.694996 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:38.695473 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:38.696134 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:38.696140 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:38.696144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:38.696364 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:38.697216 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:50:38.697225 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:50:38.697613 (+ 388us) client_negotiation.cc:769] Sending connection context
0923 20:50:38.697900 (+ 287us) client_negotiation.cc:240] Negotiation successful
0923 20:50:38.698555 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":361,"threads_started":1}
I20260923 20:50:38.700060 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:38.673906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:50561 (local address 127.5.71.190:45223)
0923 20:50:38.674546 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:38.674566 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:38.675437 (+ 871us) server_negotiation.cc:407] Connection header received
0923 20:50:38.678668 (+ 3231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:38.678682 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:38.679182 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:38.679614 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:38.681471 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:38.682881 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:38.685355 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:38.685989 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:38.686140 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:38.691231 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:38.691244 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:38.691262 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:38.691322 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:38.694767 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:38.695667 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:38.695675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:38.695681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:38.695979 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:38.696501 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:38.696507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:38.696510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:38.697045 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:38.697292 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:38.698152 (+ 860us) server_negotiation.cc:299] Negotiation successful
0923 20:50:38.698630 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":229,"threads_started":1}
I20260923 20:50:38.703495 21694 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45223
I20260923 20:50:38.703994 21694 heartbeater.cc:464] Registering TS with master...
I20260923 20:50:38.704999 21694 heartbeater.cc:511] Master 127.5.71.190:45223 requested a full tablet report, sending...
I20260923 20:50:38.707722 21509 ts_manager.cc:194] Registered new tserver with Master: e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427)
I20260923 20:50:38.709585 21509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:50561
W20260923 20:50:39.035271 21698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:39.035669 21698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:39.035907 21698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:39.044121 21698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:39.044334 21698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:39.044556 21698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:50:39.056799 21698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:35549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=39869
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:39.058483 21698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:39.060245 21698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:39.075894 21703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:39.075937 21706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:39.075932 21704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:39.077044 21698 server_base.cc:1034] running on GCE node
I20260923 20:50:39.077899 21698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:39.079084 21698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:39.080358 21698 hybrid_clock.cc:648] HybridClock initialized: now 1790196639080312 us; error 46 us; skew 500 ppm
I20260923 20:50:39.080840 21698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:39 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:39.084478 21698 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:50:39.086321 21698 webserver.cc:460] Webserver started at http://127.5.71.130:39869/ using document root <none> and password file <none>
I20260923 20:50:39.087087 21698 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:39.087276 21698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:39.093732 21698 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260923 20:50:39.097468 21713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:39.099646 21698 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260923 20:50:39.100026 21698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "bcd4e89be4924cd58df6745904779990"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "5694eb14b8ba0765624dda7372a7c1b7"
server_key_iv: "6e3686223e520879c46a4100b484a998"
server_key_version: "encryptionkey@0"
I20260923 20:50:39.101243 21698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:39.120874 21698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:39.126631 21698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:39.127017 21698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:39.128600 21698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:50:39.130468 21698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:50:39.130568 21698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:39.130649 21698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:50:39.130748 21698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:39.171088 21698 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:35549
I20260923 20:50:39.171144 21825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:35549 every 8 connection(s)
I20260923 20:50:39.172683 21698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:50:39.176366 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21698
I20260923 20:50:39.177276 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:35435
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:45223
--webserver_port=33695
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 20:50:39 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:39.198541 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.175574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:60773 (local address 127.5.71.190:45223)
0923 20:50:39.175868 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.175872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.177451 (+ 1579us) server_negotiation.cc:407] Connection header received
0923 20:50:39.180605 (+ 3154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.180613 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.180819 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.180933 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:39.183319 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.184373 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.186612 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.187077 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.187230 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.191355 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:39.191360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:39.191364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:39.191401 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:39.194430 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.195527 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.195532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.195534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.195714 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.196394 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.196399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.196401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.196763 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:39.196978 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.197871 (+ 893us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.198137 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260923 20:50:39.199671 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.176241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:60773)
0923 20:50:39.177264 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:39.177320 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:39.180230 (+ 2910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:39.181248 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:39.181263 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:39.182093 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:50:39.183078 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.183111 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.184561 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.184569 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.186425 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.186439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.187338 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.187345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.187585 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.188384 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:50:39.188408 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:50:39.191075 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:50:39.194605 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:39.194618 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:39.194650 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:39.195350 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:39.195932 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:39.195940 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:39.195947 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:39.196227 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:39.196953 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:50:39.196962 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:50:39.197450 (+ 488us) client_negotiation.cc:769] Sending connection context
0923 20:50:39.197782 (+ 332us) client_negotiation.cc:240] Negotiation successful
0923 20:50:39.198410 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":433,"threads_started":1}
I20260923 20:50:39.201510 21826 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45223
I20260923 20:50:39.201910 21826 heartbeater.cc:464] Registering TS with master...
I20260923 20:50:39.202714 21826 heartbeater.cc:511] Master 127.5.71.190:45223 requested a full tablet report, sending...
I20260923 20:50:39.204401 21509 ts_manager.cc:194] Registered new tserver with Master: bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549)
I20260923 20:50:39.205286 21509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:60773
W20260923 20:50:39.531641 21829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:50:39.532033 21829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:50:39.532213 21829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:50:39.539688 21829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:50:39.539911 21829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:50:39.540098 21829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:50:39.550661 21829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:35435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=33695
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:50:39.552271 21829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:50:39.553683 21829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:50:39.569425 21834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:39.569233 21835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:50:39.569290 21837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:50:39.570703 21829 server_base.cc:1034] running on GCE node
I20260923 20:50:39.571414 21829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:50:39.572506 21829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:50:39.573762 21829 hybrid_clock.cc:648] HybridClock initialized: now 1790196639573690 us; error 37 us; skew 500 ppm
I20260923 20:50:39.574082 21829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:50:39 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:50:39.577582 21829 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:50:39.579312 21829 webserver.cc:460] Webserver started at http://127.5.71.131:33695/ using document root <none> and password file <none>
I20260923 20:50:39.580123 21829 fs_manager.cc:359] Metadata directory not provided
I20260923 20:50:39.580304 21829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:50:39.586676 21829 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260923 20:50:39.590221 21844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:39.592227 21829 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:50:39.592396 21829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5"
format_stamp: "Formatted at 2026-09-23 20:50:16 on dist-test-slave-dn29"
server_key: "55afd6600e135c1189cc4b9d5a14c9d3"
server_key_iv: "596da04367d42420141b12f9fc49646f"
server_key_version: "encryptionkey@0"
I20260923 20:50:39.593503 21829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:50:39.621929 21829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:50:39.627013 21829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:50:39.627265 21829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:50:39.628330 21829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:50:39.629737 21829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:50:39.629834 21829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:39.629905 21829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:50:39.629961 21829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:50:39.667138 21829 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:35435
I20260923 20:50:39.667204 21956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:35435 every 8 connection(s)
I20260923 20:50:39.668834 21829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:50:39.677605 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 21829
Sep 23 20:50:39 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196639, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:50:39.693450 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.671894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:45165 (local address 127.5.71.190:45223)
0923 20:50:39.672149 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.672157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.673385 (+ 1228us) server_negotiation.cc:407] Connection header received
0923 20:50:39.676200 (+ 2815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.676205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.676289 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.676386 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:39.678151 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.679213 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.681309 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.682141 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.682321 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.686735 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:39.686739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:39.686743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:39.686775 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:39.689928 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.690748 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.690756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.690763 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.690923 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.691383 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.691388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.691391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.691877 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:39.692043 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.692919 (+ 876us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.693146 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260923 20:50:39.694875 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.672282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:45165)
0923 20:50:39.673170 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:39.673236 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:39.675880 (+ 2644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:39.676629 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:39.676639 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:39.677232 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:50:39.677940 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.677962 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.679451 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.679455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.681067 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.681080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.682395 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.682400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.682769 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.683605 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:50:39.683643 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:50:39.686472 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:50:39.690083 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:39.690100 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:39.690116 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:39.690585 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:39.691039 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:50:39.691042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:50:39.691044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:50:39.691268 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:50:39.692046 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:50:39.692062 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:50:39.692427 (+ 365us) client_negotiation.cc:769] Sending connection context
0923 20:50:39.692767 (+ 340us) client_negotiation.cc:240] Negotiation successful
0923 20:50:39.693260 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":349,"threads_started":1}
I20260923 20:50:39.696781 21957 heartbeater.cc:347] Connected to a master server at 127.5.71.190:45223
I20260923 20:50:39.697170 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.678764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.5.71.190:45223)
0923 20:50:39.679363 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.679373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.679744 (+ 371us) server_negotiation.cc:407] Connection header received
0923 20:50:39.680050 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.680056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.680137 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.680252 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:39.682522 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.683590 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.685634 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.686134 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.686236 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.689562 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:39.689568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:39.689573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:39.689633 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:39.693066 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.694020 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.694024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.694026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.694143 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.694724 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.694729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.694731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.695614 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:39.695904 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.696486 (+ 582us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.696772 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":279,"threads_started":1}
I20260923 20:50:39.697176 21957 heartbeater.cc:464] Registering TS with master...
I20260923 20:50:39.698145 21957 heartbeater.cc:511] Master 127.5.71.190:45223 requested a full tablet report, sending...
I20260923 20:50:39.699967 21508 ts_manager.cc:194] Registered new tserver with Master: 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
I20260923 20:50:39.700939 21508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:45165
I20260923 20:50:39.713300 21694 heartbeater.cc:503] Master 127.5.71.190:45223 was elected leader, sending a full tablet report...
I20260923 20:50:39.713336 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:50:39.721437 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.714581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.5.71.190:45223)
0923 20:50:39.714890 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.714895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.715359 (+ 464us) server_negotiation.cc:407] Connection header received
0923 20:50:39.715648 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.715652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.715896 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.716082 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:50:39.716597 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.717356 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.718932 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.719416 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.719610 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.720298 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:50:39.720555 (+ 257us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 20:50:39.721060 (+ 505us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34964: token expired
Metrics: {"server-negotiator.queue_time_us":158}
W20260923 20:50:39.721655 21960 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34964: token expired
W20260923 20:50:39.722244 21431 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.71.190:45223)
I20260923 20:50:39.737447 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.722855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.5.71.190:45223)
0923 20:50:39.723066 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.723074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.723190 (+ 116us) server_negotiation.cc:407] Connection header received
0923 20:50:39.723394 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.723401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.723469 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.723579 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:39.725134 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.725886 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.727436 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.727942 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.728085 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.731097 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:39.731103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:39.731106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:39.731144 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:39.734109 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.735005 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.735012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.735016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.735177 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:39.735643 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:39.735651 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:39.735655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:39.736109 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:39.736408 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.736795 (+ 387us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.737069 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 20:50:39.741912 21508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:50:39.749864 21508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:50:39.781106 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.763807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:35549 (local address 127.0.0.1:41766)
0923 20:50:39.767533 (+ 3726us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:39.767590 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:39.767853 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:39.769704 (+ 1851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:39.769712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:39.769746 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:39.770317 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.770336 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.773031 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.773041 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.776673 (+ 3632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.776708 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.779751 (+ 3043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.779759 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.780131 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.780212 (+ 81us) client_negotiation.cc:769] Sending connection context
0923 20:50:39.780360 (+ 148us) client_negotiation.cc:240] Negotiation successful
0923 20:50:39.780719 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3356,"thread_start_us":16426,"threads_started":1}
I20260923 20:50:39.782377 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.763844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.5.71.130:35549)
0923 20:50:39.768977 (+ 5133us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.768992 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.769019 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:50:39.769098 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.769104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.769400 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.769690 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:39.770600 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.772795 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.777054 (+ 4259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.779391 (+ 2337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.781434 (+ 2043us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.781728 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.781809 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.782016 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4983,"thread_start_us":4880,"threads_started":1}
I20260923 20:50:39.787179 21761 tablet_service.cc:1467] Processing CreateTablet for tablet 40fe556c05534286859e13ff2160ed4d (DEFAULT_TABLE table=test-table [id=aacb29fb2dda4c388a09b97f74014b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:50:39.788928 21761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40fe556c05534286859e13ff2160ed4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:50:39.789372 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.764154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34252)
0923 20:50:39.780414 (+ 16260us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:39.780437 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:39.780589 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:39.782075 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:39.782081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:39.782113 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:39.782482 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.782628 (+ 146us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.784416 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.784425 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.786152 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.786166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.788537 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.788543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.788708 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.788741 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 20:50:39.788840 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 20:50:39.788994 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16063,"thread_start_us":2791,"threads_started":1}
I20260923 20:50:39.790156 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.764917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.5.71.129:35427)
0923 20:50:39.780184 (+ 15267us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.780199 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.780728 (+ 529us) server_negotiation.cc:407] Connection header received
0923 20:50:39.780851 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.780863 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.781543 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.781785 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:39.782781 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.784229 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.786353 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.787957 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.788107 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.788357 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.789640 (+ 1283us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.789833 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14887,"thread_start_us":14586,"threads_started":1}
I20260923 20:50:39.791647 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.769799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:35435 (local address 127.0.0.1:58054)
0923 20:50:39.780885 (+ 11086us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:39.780912 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:39.781102 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:39.783446 (+ 2344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:39.783451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:39.783480 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:39.783846 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.783858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.786420 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.786427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.788418 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:39.788434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.790801 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.790807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:39.791003 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.791035 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:50:39.791120 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 20:50:39.791268 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10862,"thread_start_us":11485,"threads_started":1}
I20260923 20:50:39.792048 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:39.770179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.5.71.131:35435)
0923 20:50:39.782698 (+ 12519us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:39.782709 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:39.782751 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:50:39.782865 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:39.782875 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:39.783194 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:39.783424 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:39.784323 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.786241 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.788629 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:39.790259 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:39.790393 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:39.790650 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:39.791447 (+ 797us) server_negotiation.cc:299] Negotiation successful
0923 20:50:39.791624 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12289,"thread_start_us":12084,"threads_started":1}
I20260923 20:50:39.794817 21629 tablet_service.cc:1467] Processing CreateTablet for tablet 40fe556c05534286859e13ff2160ed4d (DEFAULT_TABLE table=test-table [id=aacb29fb2dda4c388a09b97f74014b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:50:39.796576 21629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40fe556c05534286859e13ff2160ed4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:50:39.796555 21892 tablet_service.cc:1467] Processing CreateTablet for tablet 40fe556c05534286859e13ff2160ed4d (DEFAULT_TABLE table=test-table [id=aacb29fb2dda4c388a09b97f74014b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:50:39.798311 21892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40fe556c05534286859e13ff2160ed4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:50:39.804582 21973 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Bootstrap starting.
I20260923 20:50:39.808746 21973 tablet_bootstrap.cc:654] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:39.809973 21973 log.cc:824] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:39.816967 21975 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Bootstrap starting.
I20260923 20:50:39.818914 21976 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Bootstrap starting.
I20260923 20:50:39.821208 21973 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: No bootstrap required, opened a new log
I20260923 20:50:39.821646 21973 ts_tablet_manager.cc:1397] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.009s
I20260923 20:50:39.822006 21975 tablet_bootstrap.cc:654] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:39.823060 21976 tablet_bootstrap.cc:654] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Neither blocks nor log segments found. Creating new log.
I20260923 20:50:39.823468 21975 log.cc:824] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:39.824667 21976 log.cc:824] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Log is configured to *not* fsync() on all Append() calls
I20260923 20:50:39.828334 21976 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: No bootstrap required, opened a new log
I20260923 20:50:39.828648 21976 ts_tablet_manager.cc:1397] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260923 20:50:39.829609 21975 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: No bootstrap required, opened a new log
I20260923 20:50:39.829907 21975 ts_tablet_manager.cc:1397] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260923 20:50:39.833134 21973 raft_consensus.cc:348] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:39.833626 21973 raft_consensus.cc:374] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:39.833715 21973 raft_consensus.cc:729] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd4e89be4924cd58df6745904779990, State: Initialized, Role: FOLLOWER
I20260923 20:50:39.834378 21973 consensus_queue.cc:260] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:39.836308 21826 heartbeater.cc:503] Master 127.5.71.190:45223 was elected leader, sending a full tablet report...
I20260923 20:50:39.836999 21973 ts_tablet_manager.cc:1428] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260923 20:50:39.839478 21975 raft_consensus.cc:348] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:39.839493 21976 raft_consensus.cc:348] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:39.839946 21975 raft_consensus.cc:374] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:39.840004 21976 raft_consensus.cc:374] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:50:39.840092 21975 raft_consensus.cc:729] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44dec8a18b6439da47ceb3945af781a, State: Initialized, Role: FOLLOWER
I20260923 20:50:39.840112 21976 raft_consensus.cc:729] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, State: Initialized, Role: FOLLOWER
I20260923 20:50:39.840806 21976 consensus_queue.cc:260] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:39.840787 21975 consensus_queue.cc:260] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:39.841979 21957 heartbeater.cc:503] Master 127.5.71.190:45223 was elected leader, sending a full tablet report...
I20260923 20:50:39.842736 21976 ts_tablet_manager.cc:1428] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
I20260923 20:50:39.844205 21975 ts_tablet_manager.cc:1428] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
W20260923 20:50:39.921074 21958 tablet.cc:2369] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:50:39.923357 21695 tablet.cc:2369] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:50:39.924898 21827 tablet.cc:2369] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:50:40.033885 21981 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:50:40.034348 21981 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:40.036554 21981 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
I20260923 20:50:40.043813 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.037088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:48983 (local address 127.5.71.131:35435)
0923 20:50:40.037376 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.037383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.038213 (+ 830us) server_negotiation.cc:407] Connection header received
0923 20:50:40.038437 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.038442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.038501 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.038609 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:40.039250 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.040042 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.041982 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.042774 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.042990 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.043072 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.043364 (+ 292us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.043495 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260923 20:50:40.043964 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.037149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:35435 (local address 127.5.71.129:48983)
0923 20:50:40.038038 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.038069 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:40.038264 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:40.038626 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:40.038631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:40.038654 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:40.039081 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.039091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.040192 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.040198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.041798 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.041812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.043001 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.043007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.043183 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.043208 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 20:50:40.043340 (+ 132us) client_negotiation.cc:240] Negotiation successful
0923 20:50:40.043488 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":4,"thread_start_us":474,"threads_started":1}
I20260923 20:50:40.044176 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.037219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:55213 (local address 127.5.71.130:35549)
0923 20:50:40.037499 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.037509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.038385 (+ 876us) server_negotiation.cc:407] Connection header received
0923 20:50:40.038611 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.038617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.038683 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.038803 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:40.039462 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.040298 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.042197 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.043068 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.043288 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.043378 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.043708 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.043885 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260923 20:50:40.044260 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.037346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:35549 (local address 127.5.71.129:55213)
0923 20:50:40.038238 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.038255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:40.038432 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:40.038822 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:40.038826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:40.038857 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:40.039275 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.039285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.040478 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.040490 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.042005 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.042018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.043329 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.043336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.043509 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.043539 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 20:50:40.043680 (+ 141us) client_negotiation.cc:240] Negotiation successful
0923 20:50:40.043851 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":535,"threads_started":1}
I20260923 20:50:40.045186 21912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fe556c05534286859e13ff2160ed4d" candidate_uuid: "e44dec8a18b6439da47ceb3945af781a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" is_pre_election: true
I20260923 20:50:40.045377 21781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fe556c05534286859e13ff2160ed4d" candidate_uuid: "e44dec8a18b6439da47ceb3945af781a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e89be4924cd58df6745904779990" is_pre_election: true
I20260923 20:50:40.045874 21781 raft_consensus.cc:2454] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44dec8a18b6439da47ceb3945af781a in term 0.
I20260923 20:50:40.045946 21912 raft_consensus.cc:2454] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44dec8a18b6439da47ceb3945af781a in term 0.
I20260923 20:50:40.046221 21980 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:50:40.046509 21980 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:40.046710 21582 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, e44dec8a18b6439da47ceb3945af781a; no voters:
I20260923 20:50:40.047170 21981 raft_consensus.cc:2781] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:50:40.047344 21981 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:50:40.047458 21981 raft_consensus.cc:3037] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:40.048547 21980 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427)
I20260923 20:50:40.050876 21981 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:40.051733 21981 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 1 election: Requested vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
I20260923 20:50:40.052178 21781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fe556c05534286859e13ff2160ed4d" candidate_uuid: "e44dec8a18b6439da47ceb3945af781a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e89be4924cd58df6745904779990"
I20260923 20:50:40.052430 21781 raft_consensus.cc:3037] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:40.052486 21912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fe556c05534286859e13ff2160ed4d" candidate_uuid: "e44dec8a18b6439da47ceb3945af781a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5"
I20260923 20:50:40.052748 21912 raft_consensus.cc:3037] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:50:40.055989 21781 raft_consensus.cc:2454] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44dec8a18b6439da47ceb3945af781a in term 1.
I20260923 20:50:40.056202 21912 raft_consensus.cc:2454] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44dec8a18b6439da47ceb3945af781a in term 1.
I20260923 20:50:40.056965 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.048888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:35549 (local address 127.5.71.131:48241)
0923 20:50:40.049248 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.049269 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:40.049388 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:40.049962 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:40.049967 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:40.049997 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:40.050203 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.050225 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.051776 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.051807 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.054223 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.054239 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.056132 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.056139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.056352 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.056396 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 20:50:40.056488 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 20:50:40.056633 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":24}
I20260923 20:50:40.056912 21582 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990, e44dec8a18b6439da47ceb3945af781a; no voters:
I20260923 20:50:40.057348 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.049607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:56903 (local address 127.5.71.129:35427)
0923 20:50:40.049806 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.049813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.049830 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:50:40.050012 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.050017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.050105 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.050165 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:40.050771 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.051706 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.054720 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.056054 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.056190 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.056280 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.056980 (+ 700us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.057131 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 20:50:40.057513 21981 raft_consensus.cc:2781] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:50:40.058034 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.049012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:48241 (local address 127.5.71.130:35549)
0923 20:50:40.049637 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.049643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.049662 (+ 19us) server_negotiation.cc:407] Connection header received
0923 20:50:40.049723 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.049729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.049817 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.050346 (+ 529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:50:40.050481 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.051611 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.054538 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.055907 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.057458 (+ 1551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.057547 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.057601 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.057741 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497}
I20260923 20:50:40.058065 21981 raft_consensus.cc:686] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 LEADER]: Becoming Leader. State: Replica: e44dec8a18b6439da47ceb3945af781a, State: Running, Role: LEADER
I20260923 20:50:40.058234 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.049000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.5.71.131:56903)
0923 20:50:40.049728 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.049747 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:50:40.049861 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:50:40.050307 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:50:40.050310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:50:40.050328 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:50:40.050612 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.050619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.052024 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.052031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.054490 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:50:40.054504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.056478 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.056487 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:50:40.056796 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.056838 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 20:50:40.057618 (+ 780us) client_negotiation.cc:240] Negotiation successful
0923 20:50:40.057778 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":3,"thread_start_us":898,"threads_started":1}
I20260923 20:50:40.058600 21781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fe556c05534286859e13ff2160ed4d" candidate_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd4e89be4924cd58df6745904779990" is_pre_election: true
I20260923 20:50:40.058943 21781 raft_consensus.cc:2379] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e5a5e3f6cd64ceca18e4ace46f3b7e5 in current term 1: Already voted for candidate e44dec8a18b6439da47ceb3945af781a in this term.
I20260923 20:50:40.058759 21981 consensus_queue.cc:237] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:50:40.059123 21649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fe556c05534286859e13ff2160ed4d" candidate_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44dec8a18b6439da47ceb3945af781a" is_pre_election: true
I20260923 20:50:40.060246 21846 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5; no voters: bcd4e89be4924cd58df6745904779990, e44dec8a18b6439da47ceb3945af781a
I20260923 20:50:40.060707 21980 raft_consensus.cc:2726] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 20:50:40.064800 21508 catalog_manager.cc:5495] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a reported cstate change: term changed from 0 to 1, leader changed from <none> to e44dec8a18b6439da47ceb3945af781a (127.5.71.129). New cstate: current_term: 1 leader_uuid: "e44dec8a18b6439da47ceb3945af781a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } health_report { overall_health: UNKNOWN } } }
I20260923 20:50:40.097609 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.090915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.5.71.129:35427)
0923 20:50:40.091146 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.091152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.091173 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:50:40.091412 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.091417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.091518 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.091668 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:50:40.092271 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.093093 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.094764 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.095308 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.095423 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.096276 (+ 853us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:50:40.096890 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:50:40.097093 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.097170 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.097270 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 20:50:40.107198 21781 raft_consensus.cc:1261] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Refusing update from remote peer e44dec8a18b6439da47ceb3945af781a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:50:40.107406 21912 raft_consensus.cc:1261] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Refusing update from remote peer e44dec8a18b6439da47ceb3945af781a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:50:40.108389 21985 consensus_queue.cc:1035] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:50:40.108798 21981 consensus_queue.cc:1035] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:50:40.124764 21989 mvcc.cc:205] Tried to move back new op lower bound from 7332645437868019712 to 7332645437686087680. Current Snapshot: MvccSnapshot[applied={T|T < 7332645437868019712}]
I20260923 20:50:40.145972 21509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:50:40.146297 21509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:50:40.150919 21509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Sending DeleteTablet for 3 replicas of tablet 40fe556c05534286859e13ff2160ed4d
I20260923 20:50:40.152323 21629 tablet_service.cc:1514] Processing DeleteTablet for tablet 40fe556c05534286859e13ff2160ed4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:50:40 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:34252
I20260923 20:50:40.152356 21892 tablet_service.cc:1514] Processing DeleteTablet for tablet 40fe556c05534286859e13ff2160ed4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:50:40 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:58054
I20260923 20:50:40.152954 21761 tablet_service.cc:1514] Processing DeleteTablet for tablet 40fe556c05534286859e13ff2160ed4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:50:40 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:41766
I20260923 20:50:40.154013 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21546
I20260923 20:50:40.164579 22000 tablet_replica.cc:330] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: stopping tablet replica
I20260923 20:50:40.165163 22000 raft_consensus.cc:2229] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:50:40.165266 21998 tablet_replica.cc:330] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: stopping tablet replica
I20260923 20:50:40.165755 22000 raft_consensus.cc:2258] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:50:40.166006 21998 raft_consensus.cc:2229] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:50:40.166566 21998 raft_consensus.cc:2258] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:50:40.169093 21998 ts_tablet_manager.cc:1905] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:50:40.169952 22000 ts_tablet_manager.cc:1905] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:50:40.172793 21999 tablet_replica.cc:330] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: stopping tablet replica
I20260923 20:50:40.173553 21999 raft_consensus.cc:2229] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:50:40.174319 21999 raft_consensus.cc:2258] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:50:40.178642 22000 ts_tablet_manager.cc:1918] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:50:40.178838 22000 log.cc:1196] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/40fe556c05534286859e13ff2160ed4d
I20260923 20:50:40.179478 22000 ts_tablet_manager.cc:1939] T 40fe556c05534286859e13ff2160ed4d P bcd4e89be4924cd58df6745904779990: Deleting consensus metadata
I20260923 20:50:40.181676 21495 catalog_manager.cc:4841] TS bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): tablet 40fe556c05534286859e13ff2160ed4d (table test-table [id=aacb29fb2dda4c388a09b97f74014b8d]) successfully deleted
I20260923 20:50:40.182153 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.155958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34264 (local address 127.5.71.129:35427)
0923 20:50:40.156201 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.156210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.156238 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:50:40.156319 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.156325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.156485 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.156936 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:40.159704 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.161464 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.165012 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.165700 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.165875 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.170849 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:40.170866 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:40.170885 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:40.170933 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:40.176877 (+ 5944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:40.177968 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:40.177977 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:40.177981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:40.178152 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:40.178682 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:40.178687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:40.178690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:40.179402 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:40.181448 (+ 2046us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.181603 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.181797 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260923 20:50:40.182942 21689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:40.190794 21998 ts_tablet_manager.cc:1918] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:50:40.190973 21998 log.cc:1196] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/40fe556c05534286859e13ff2160ed4d
I20260923 20:50:40.191598 21998 ts_tablet_manager.cc:1939] T 40fe556c05534286859e13ff2160ed4d P 5e5a5e3f6cd64ceca18e4ace46f3b7e5: Deleting consensus metadata
I20260923 20:50:40.193657 21493 catalog_manager.cc:4841] TS 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): tablet 40fe556c05534286859e13ff2160ed4d (table test-table [id=aacb29fb2dda4c388a09b97f74014b8d]) successfully deleted
I20260923 20:50:40.184306 21999 ts_tablet_manager.cc:1905] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:50:40.286590 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21546
W20260923 20:50:40.308167 21494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:35427 (error 108)
I20260923 20:50:40.308362 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21698
I20260923 20:50:40.310741 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.309840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34276)
0923 20:50:40.310334 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.310451 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":262,"threads_started":1}
W20260923 20:50:40.311127 21494 catalog_manager.cc:4584] TS e44dec8a18b6439da47ceb3945af781a (127.5.71.129:35427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40fe556c05534286859e13ff2160ed4d: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
I20260923 20:50:40.325170 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.309315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.5.71.130:35549)
0923 20:50:40.309536 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.309540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.309560 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:50:40.309627 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.309650 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.309724 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.309815 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:40.311570 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.312497 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.314082 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.314743 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.314878 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.317775 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:40.317790 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:40.317809 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:40.317849 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:40.321346 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:40.322347 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:40.322355 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:40.322357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:40.322528 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:40.323177 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:40.323182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:40.323184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:40.323683 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:40.324086 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.324549 (+ 463us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.324803 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 20:50:40.326058 21821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:40.365052 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.364561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34282)
0923 20:50:40.364784 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.364900 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260923 20:50:40.366284 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.365846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34288)
0923 20:50:40.366018 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.366125 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260923 20:50:40.419199 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21698
I20260923 20:50:40.427312 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.426778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34290)
0923 20:50:40.427002 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.427154 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260923 20:50:40.428925 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.428527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34302)
0923 20:50:40.428680 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.428767 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260923 20:50:40.438211 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21829
I20260923 20:50:40.453621 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.439051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.5.71.131:35435)
0923 20:50:40.439278 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0923 20:50:40.439285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:50:40.439307 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:50:40.439539 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:50:40.439544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:50:40.439612 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:50:40.439724 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:50:40.441179 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.441961 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.443346 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:50:40.443756 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:50:40.443967 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:50:40.446691 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:50:40.446708 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:50:40.446727 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:50:40.446782 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:50:40.450207 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:40.451078 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:40.451087 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:40.451090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:40.451248 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:50:40.451699 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:50:40.451705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:50:40.451708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:50:40.452243 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:50:40.452591 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0923 20:50:40.452954 (+ 363us) server_negotiation.cc:299] Negotiation successful
0923 20:50:40.453203 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:50:40.454365 21952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:40.509922 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.509324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34314)
0923 20:50:40.509555 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.509756 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260923 20:50:40.511063 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.510624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34330)
0923 20:50:40.510843 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.510914 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260923 20:50:40.559899 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21829
I20260923 20:50:40.579445 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 21476
I20260923 20:50:40.581073 21539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:50:40.700999 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:50:40.700188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:35427 (local address 127.0.0.1:34340)
0923 20:50:40.700534 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0923 20:50:40.700691 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:35427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260923 20:50:40.701026 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 21476
I20260923 20:51:00.726322 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45223
--webserver_interface=127.5.71.190
--webserver_port=43961
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:01.083297 22007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:01.083643 22007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:01.083844 22007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:01.091666 22007 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 20:51:01.091854 22007 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 20:51:01.091926 22007 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 20:51:01.092006 22007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:01.092103 22007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:01.092217 22007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:01.092288 22007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:01.092370 22007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:01.103895 22007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=43961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:01.105494 22007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:01.107196 22007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:01.121836 22015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:01.121919 22007 server_base.cc:1034] running on GCE node
W20260923 20:51:01.122291 22012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:01.122365 22013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:01.123415 22007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:01.125124 22007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:01.126384 22007 hybrid_clock.cc:648] HybridClock initialized: now 1790196661126308 us; error 42 us; skew 500 ppm
I20260923 20:51:01.126833 22007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:01 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:01.136914 22007 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:51:01.138622 22007 webserver.cc:460] Webserver started at http://127.5.71.190:43961/ using document root <none> and password file <none>
I20260923 20:51:01.139418 22007 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:01.139516 22007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:01.145717 22007 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:51:01.149055 22022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:01.150714 22007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260923 20:51:01.150944 22007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "1d272365f98c4385ad88884b7b4f973a"
format_stamp: "Formatted at 2026-09-23 20:50:14 on dist-test-slave-dn29"
server_key: "9d1d43f9a036b7d86088900ddae582ce"
server_key_iv: "19c231d3db3d2b8868917c0fdece368b"
server_key_version: "encryptionkey@0"
I20260923 20:51:01.152153 22007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:01.168416 22007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:01.173380 22007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:01.173708 22007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:01.194571 22007 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45223
I20260923 20:51:01.194603 22073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45223 every 8 connection(s)
I20260923 20:51:01.196545 22007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:01.203464 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22007
I20260923 20:51:01.204077 22074 sys_catalog.cc:263] Verifying existing consensus state
I20260923 20:51:01.204432 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:35427
--local_ip_for_outbound_sockets=127.5.71.129
--tserver_master_addrs=127.5.71.190:45223
--webserver_port=43111
--webserver_interface=127.5.71.129
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:51:01.207626 22074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap starting.
I20260923 20:51:01.230885 22074 log.cc:824] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:01.248406 22074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:51:01.249011 22074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Bootstrap complete.
I20260923 20:51:01.258862 22074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:51:01.259902 22074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d272365f98c4385ad88884b7b4f973a, State: Initialized, Role: FOLLOWER
I20260923 20:51:01.260632 22074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:51:01.260890 22074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:01.261018 22074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:01.261209 22074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 2 FOLLOWER]: Advancing to term 3
I20260923 20:51:01.265461 22074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:51:01.266108 22074 leader_election.cc:302] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d272365f98c4385ad88884b7b4f973a; no voters:
I20260923 20:51:01.266785 22074 leader_election.cc:288] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 20:51:01.267001 22078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 3 FOLLOWER]: Leader election won for term 3
I20260923 20:51:01.268172 22078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [term 3 LEADER]: Becoming Leader. State: Replica: 1d272365f98c4385ad88884b7b4f973a, State: Running, Role: LEADER
I20260923 20:51:01.268996 22074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:01.268855 22078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } }
I20260923 20:51:01.271385 22080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d272365f98c4385ad88884b7b4f973a. Latest consensus state: current_term: 3 leader_uuid: "1d272365f98c4385ad88884b7b4f973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } } }
I20260923 20:51:01.271749 22080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:01.276257 22079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1d272365f98c4385ad88884b7b4f973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d272365f98c4385ad88884b7b4f973a" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45223 } } }
I20260923 20:51:01.276607 22079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:01.279166 22083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:01.288704 22083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:01.290269 22083 catalog_manager.cc:1259] Loaded cluster ID: b0e56be013684cc4affc3c3bbffc6b71
I20260923 20:51:01.290438 22083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:01.294677 22083 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:01.296957 22083 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Loaded TSK: 0
I20260923 20:51:01.297272 22083 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d272365f98c4385ad88884b7b4f973a: Loaded TSK: 1
F20260923 20:51:01.298564 22083 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3c045dd9ad google::LogMessage::Fail() at ??:0
@ 0x7f3c045e088a google::LogMessage::SendToLog() at ??:0
@ 0x7f3c045dd4de google::LogMessage::Flush() at ??:0
@ 0x7f3c045dea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3c083417d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3c0833073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3c1a00c1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3c1a00b783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3c1a0972a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f3c1a0970c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f3c1a0e15e9 std::function<>::operator()() at ??:0
@ 0x7f3c1a00fcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f3c1a00dc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3c1a095610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f3c1a095439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3c07229345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3c0722c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3c0722c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3c07207eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3c0a4e16db start_thread at ??:0
@ 0x7f3c0347761f clone at ??:0
W20260923 20:51:01.605741 22076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:01.606133 22076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:01.606328 22076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:01.614931 22076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:01.615120 22076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:01.615366 22076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:01.626565 22076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:35427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=43111
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:01.628257 22076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:01.629722 22076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:01.643914 22122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:01.643855 22125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:01.643885 22123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:01.644722 22076 server_base.cc:1034] running on GCE node
I20260923 20:51:01.645429 22076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:01.646492 22076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:01.647842 22076 hybrid_clock.cc:648] HybridClock initialized: now 1790196661647692 us; error 38 us; skew 500 ppm
I20260923 20:51:01.648268 22076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:01 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:01.651890 22076 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:51:01.653411 22076 webserver.cc:460] Webserver started at http://127.5.71.129:43111/ using document root <none> and password file <none>
I20260923 20:51:01.654088 22076 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:01.654191 22076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:01.659932 22076 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:01.663434 22132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:01.665114 22076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:01.665326 22076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "e44dec8a18b6439da47ceb3945af781a"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "c742835eec13b19572543a214f4a4ee2"
server_key_iv: "f74b961c0f9477ca93be0330b5c53019"
server_key_version: "encryptionkey@0"
I20260923 20:51:01.666455 22076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:01.682125 22076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:01.686998 22076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:01.687307 22076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:01.688383 22076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:01.690393 22139 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260923 20:51:01.696514 22076 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 20:51:01.696653 22076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260923 20:51:01.696835 22076 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 20:51:01.700450 22076 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 20:51:01.700539 22076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:01.700944 22139 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Bootstrap starting.
I20260923 20:51:01.710196 22139 log.cc:824] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:01.726550 22139 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 20:51:01.727243 22139 tablet_bootstrap.cc:492] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Bootstrap complete.
I20260923 20:51:01.727739 22139 ts_tablet_manager.cc:1397] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.008s
I20260923 20:51:01.737044 22139 raft_consensus.cc:348] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:01.738648 22139 raft_consensus.cc:729] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44dec8a18b6439da47ceb3945af781a, State: Initialized, Role: FOLLOWER
I20260923 20:51:01.739522 22139 consensus_queue.cc:260] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:01.741652 22139 ts_tablet_manager.cc:1428] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260923 20:51:01.749791 22076 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:35427
I20260923 20:51:01.749877 22247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:35427 every 8 connection(s)
I20260923 20:51:01.751111 22076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:01.756193 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:01.754017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:43725)
0923 20:51:01.754934 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:01.755212 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":346,"threads_started":1}
W20260923 20:51:01.757191 22248 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45223 (0 consecutive failures): Network error: Failed to ping master at 127.5.71.190:45223: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
I20260923 20:51:01.758226 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:01.757868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:33795)
0923 20:51:01.758042 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:01.758121 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260923 20:51:01.759256 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:01.758908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:45885)
0923 20:51:01.759064 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:01.759138 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260923 20:51:01.759634 22248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 20:51:01.761457 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22076
I20260923 20:51:01.762107 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:35549
--local_ip_for_outbound_sockets=127.5.71.130
--tserver_master_addrs=127.5.71.190:45223
--webserver_port=39869
--webserver_interface=127.5.71.130
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:02.108145 22251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:02.108479 22251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:02.108579 22251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:02.116596 22251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:02.116736 22251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:02.116855 22251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:02.127437 22251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:35549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=39869
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:02.128842 22251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:02.130194 22251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:02.144321 22256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:02.144665 22257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:02.144672 22259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:02.145674 22251 server_base.cc:1034] running on GCE node
I20260923 20:51:02.146385 22251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:02.147428 22251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:02.148691 22251 hybrid_clock.cc:648] HybridClock initialized: now 1790196662148640 us; error 43 us; skew 500 ppm
I20260923 20:51:02.149049 22251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:02 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196662, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:02.152287 22251 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:51:02.153927 22251 webserver.cc:460] Webserver started at http://127.5.71.130:39869/ using document root <none> and password file <none>
I20260923 20:51:02.154603 22251 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:02.154708 22251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:02.160506 22251 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260923 20:51:02.163956 22266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:02.165534 22251 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:02.165728 22251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "bcd4e89be4924cd58df6745904779990"
format_stamp: "Formatted at 2026-09-23 20:50:15 on dist-test-slave-dn29"
server_key: "5694eb14b8ba0765624dda7372a7c1b7"
server_key_iv: "6e3686223e520879c46a4100b484a998"
server_key_version: "encryptionkey@0"
I20260923 20:51:02.166785 22251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:02.186504 22251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:02.191676 22251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:02.192061 22251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:02.193140 22251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:02.194563 22251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:02.194658 22251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:02.194828 22251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:02.194936 22251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:02.231456 22251 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:35549
I20260923 20:51:02.231545 22378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:35549 every 8 connection(s)
I20260923 20:51:02.233039 22251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:02.238250 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22251
I20260923 20:51:02.238308 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.236129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:59441)
0923 20:51:02.237119 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.237344 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":422,"threads_started":1}
I20260923 20:51:02.239074 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:35435
--local_ip_for_outbound_sockets=127.5.71.131
--tserver_master_addrs=127.5.71.190:45223
--webserver_port=33695
--webserver_interface=127.5.71.131
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:02.239164 22379 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45223 (0 consecutive failures): Network error: Failed to ping master at 127.5.71.190:45223: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
I20260923 20:51:02.240212 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.239747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:41027)
0923 20:51:02.239952 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.240036 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260923 20:51:02.241842 22381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.241387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:54581)
0923 20:51:02.241577 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.241682 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260923 20:51:02.242125 22379 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 20:51:02.588626 22382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:02.588990 22382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:02.589167 22382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:02.597321 22382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:02.597550 22382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:02.597764 22382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:02.608283 22382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:35435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=33695
--tserver_master_addrs=127.5.71.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:02.609767 22382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:02.611181 22382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:02.626222 22388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:02.626255 22387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:02.626312 22390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:02.627403 22382 server_base.cc:1034] running on GCE node
I20260923 20:51:02.628173 22382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:02.629251 22382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:02.630520 22382 hybrid_clock.cc:648] HybridClock initialized: now 1790196662630416 us; error 94 us; skew 500 ppm
I20260923 20:51:02.631017 22382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:02 dist-test-slave-dn29 krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196662, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:02.634774 22382 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:51:02.636749 22382 webserver.cc:460] Webserver started at http://127.5.71.131:33695/ using document root <none> and password file <none>
I20260923 20:51:02.637557 22382 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:02.637735 22382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:02.644232 22382 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:02.647392 22397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:02.649221 22382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:02.649461 22382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5"
format_stamp: "Formatted at 2026-09-23 20:50:16 on dist-test-slave-dn29"
server_key: "55afd6600e135c1189cc4b9d5a14c9d3"
server_key_iv: "596da04367d42420141b12f9fc49646f"
server_key_version: "encryptionkey@0"
I20260923 20:51:02.650535 22382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:02.667589 22382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:02.673513 22382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:02.673838 22382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:02.674988 22382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:02.676529 22382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:02.676669 22382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:02.676807 22382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:02.676896 22382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:02.718994 22382 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:35435
I20260923 20:51:02.719048 22509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:35435 every 8 connection(s)
I20260923 20:51:02.720700 22382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:02.726837 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22382
I20260923 20:51:02.726892 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.724525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:33003)
0923 20:51:02.725448 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.725679 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":349,"threads_started":1}
W20260923 20:51:02.727677 22510 heartbeater.cc:644] Failed to heartbeat to 127.5.71.190:45223 (0 consecutive failures): Network error: Failed to ping master at 127.5.71.190:45223: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
I20260923 20:51:02.728730 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.728294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:34159)
0923 20:51:02.728485 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.728576 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260923 20:51:02.729791 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.729398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:41671)
0923 20:51:02.729563 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.729672 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260923 20:51:02.730087 22510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 20:51:02.761133 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:02.760282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:42073)
0923 20:51:02.760793 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:02.760894 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":227,"threads_started":1}
I20260923 20:51:03.243737 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.242811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:44343)
0923 20:51:03.243289 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:03.243402 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":232,"threads_started":1}
I20260923 20:51:03.453647 22516 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:03.454031 22516 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:03.456058 22516 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
Sep 23 20:51:03 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
Sep 23 20:51:03 dist-test-slave-dn29 krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196661, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:51:03.492815 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.456589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:35435 (local address 127.5.71.129:53733)
0923 20:51:03.470151 (+ 13562us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:03.470167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:03.470285 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:03.473345 (+ 3060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:03.473355 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:03.474450 (+ 1095us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:03.475362 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:03.475383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.477026 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.477030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:03.478430 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:03.478442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.479534 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.479541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:03.479745 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:03.480613 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:03.480630 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:03.483267 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:03.488010 (+ 4743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:03.488023 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:03.488045 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:03.489013 (+ 968us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:03.489632 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:03.489638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:03.489642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:03.489972 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:03.490960 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:03.490969 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:03.491374 (+ 405us) client_negotiation.cc:769] Sending connection context
0923 20:51:03.491678 (+ 304us) client_negotiation.cc:240] Negotiation successful
0923 20:51:03.492338 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13438,"mutex_wait_us":212,"spinlock_wait_cycles":56832,"thread_start_us":13270,"threads_started":1}
I20260923 20:51:03.492805 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.456635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40633 (local address 127.5.71.130:35549)
0923 20:51:03.469430 (+ 12795us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:03.469441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:03.469674 (+ 233us) server_negotiation.cc:407] Connection header received
0923 20:51:03.472426 (+ 2752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:03.472436 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:03.472916 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:03.473243 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:03.475571 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.476821 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.478581 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.479227 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.479435 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:03.483514 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:03.483535 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:03.483551 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:03.483700 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:03.488073 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:03.489299 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:03.489307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:03.489315 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:03.489510 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:03.490285 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:03.490289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:03.490291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:03.490962 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:03.491148 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:03.491972 (+ 824us) server_negotiation.cc:299] Negotiation successful
0923 20:51:03.492378 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12513,"thread_start_us":12322,"threads_started":1}
I20260923 20:51:03.492846 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.456741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:53733 (local address 127.5.71.131:35435)
0923 20:51:03.468857 (+ 12116us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:03.468885 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:03.470294 (+ 1409us) server_negotiation.cc:407] Connection header received
0923 20:51:03.472448 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:03.472456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:03.472849 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:03.473216 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:03.475560 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.476824 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.478595 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.479287 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.479520 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:03.483557 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:03.483565 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:03.483579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:03.483626 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:03.487721 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:03.489233 (+ 1512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:03.489243 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:03.489254 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:03.489467 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:03.490118 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:03.490124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:03.490127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:03.490783 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:03.490932 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:03.492008 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0923 20:51:03.492399 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11830,"thread_start_us":11675,"threads_started":1}
I20260923 20:51:03.492933 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.456391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:35549 (local address 127.5.71.129:40633)
0923 20:51:03.469370 (+ 12979us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:03.469446 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:03.470274 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:03.473344 (+ 3070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:03.473355 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:03.474450 (+ 1095us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:03.475361 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:03.475383 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.477024 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.477028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:03.478430 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:03.478442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:03.479436 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:03.479443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:03.479709 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:03.480444 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:03.480467 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:03.483111 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:03.488375 (+ 5264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:03.488383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:03.488390 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:03.489145 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:03.489902 (+ 757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:03.489906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:03.489908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:03.490118 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:03.491113 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:03.491116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:03.491373 (+ 257us) client_negotiation.cc:769] Sending connection context
0923 20:51:03.491683 (+ 310us) client_negotiation.cc:240] Negotiation successful
0923 20:51:03.492532 (+ 849us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12769,"mutex_wait_us":325,"spinlock_wait_cycles":31488,"thread_start_us":13049,"threads_started":1}
W20260923 20:51:03.497669 22133 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
W20260923 20:51:03.498009 22135 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
I20260923 20:51:03.498183 22135 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e44dec8a18b6439da47ceb3945af781a; no voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990
I20260923 20:51:03.498638 22516 raft_consensus.cc:2726] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:51:03.731706 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.730787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:42715)
0923 20:51:03.731322 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:03.731463 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":229,"threads_started":1}
I20260923 20:51:03.762698 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:03.762164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:59799)
0923 20:51:03.762357 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:03.762477 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260923 20:51:04.245543 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:04.244754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:32803)
0923 20:51:04.245228 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:04.245355 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
I20260923 20:51:04.733582 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:04.732763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:33429)
0923 20:51:04.733288 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:04.733400 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":284,"threads_started":1}
I20260923 20:51:04.764797 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:04.763877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:53051)
0923 20:51:04.764377 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:04.764508 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":277,"threads_started":1}
I20260923 20:51:05.247334 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:05.246511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:50471)
0923 20:51:05.247064 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:05.247156 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":342,"threads_started":1}
I20260923 20:51:05.484407 22526 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:05.484589 22526 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:05.485323 22526 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
W20260923 20:51:05.486497 22135 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
W20260923 20:51:05.486671 22133 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
I20260923 20:51:05.486955 22133 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e44dec8a18b6439da47ceb3945af781a; no voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990
I20260923 20:51:05.487288 22526 raft_consensus.cc:2726] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:51:05.735553 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:05.734737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:38211)
0923 20:51:05.735283 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:05.735378 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
I20260923 20:51:05.766634 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:05.765808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:60357)
0923 20:51:05.766338 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:05.766449 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":225,"threads_started":1}
I20260923 20:51:06.249504 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:06.248408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:40525)
0923 20:51:06.249114 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:06.249236 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":384,"threads_started":1}
I20260923 20:51:06.737557 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:06.736682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:51273)
0923 20:51:06.737255 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:06.737365 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":291,"threads_started":1}
I20260923 20:51:06.768736 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:06.767858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:48977)
0923 20:51:06.768405 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:06.768512 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":256,"threads_started":1}
I20260923 20:51:07.189790 22532 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:07.190064 22532 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:07.190891 22532 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
W20260923 20:51:07.192037 22133 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
W20260923 20:51:07.192296 22135 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
I20260923 20:51:07.192510 22135 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e44dec8a18b6439da47ceb3945af781a; no voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990
I20260923 20:51:07.192899 22532 raft_consensus.cc:2726] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:51:07.251571 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:07.250693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:35699)
0923 20:51:07.251289 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:07.251384 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":346,"threads_started":1}
I20260923 20:51:07.739719 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:07.738625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:52229)
0923 20:51:07.739358 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:07.739456 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":381,"threads_started":1}
I20260923 20:51:07.770915 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:07.769844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:53437)
0923 20:51:07.770636 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:07.770724 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":367,"threads_started":1}
I20260923 20:51:08.253616 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:08.252816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:59503)
0923 20:51:08.253323 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:08.253417 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":276,"threads_started":1}
I20260923 20:51:08.741809 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:08.740811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:36589)
0923 20:51:08.741427 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:08.741523 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":293,"threads_started":1}
I20260923 20:51:08.772836 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:08.772060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:35699)
0923 20:51:08.772515 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:08.772623 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":241,"threads_started":1}
I20260923 20:51:09.260684 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:09.254766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:54649)
0923 20:51:09.260357 (+ 5591us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:09.260467 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5463,"thread_start_us":5252,"threads_started":1}
I20260923 20:51:09.744007 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:09.742984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:54955)
0923 20:51:09.743581 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:09.743720 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":299,"threads_started":1}
I20260923 20:51:09.774719 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:09.773877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:38801)
0923 20:51:09.774397 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:09.774499 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":256,"threads_started":1}
I20260923 20:51:10.262951 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:10.261903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:54007)
0923 20:51:10.262514 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:10.262726 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":307,"threads_started":1}
I20260923 20:51:10.745941 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:10.744975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:45521)
0923 20:51:10.745603 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:10.745715 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":317,"threads_started":1}
I20260923 20:51:10.776777 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:10.775861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:53037)
0923 20:51:10.776429 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:10.776522 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":274,"threads_started":1}
I20260923 20:51:11.264966 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:11.264114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:46367)
0923 20:51:11.264684 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:11.264793 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":279,"threads_started":1}
I20260923 20:51:11.536075 22546 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:11.536322 22546 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:11.537114 22546 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
W20260923 20:51:11.538149 22133 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
W20260923 20:51:11.538297 22135 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
I20260923 20:51:11.538491 22135 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e44dec8a18b6439da47ceb3945af781a; no voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990
I20260923 20:51:11.538852 22546 raft_consensus.cc:2726] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:51:11.747756 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:11.746981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:43333)
0923 20:51:11.747472 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:11.747558 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":248,"threads_started":1}
I20260923 20:51:11.779030 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:11.777874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:44747)
0923 20:51:11.778518 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:11.778774 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":285,"threads_started":1}
I20260923 20:51:12.267181 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:12.266155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:53897)
0923 20:51:12.266764 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:12.266881 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":323,"threads_started":1}
I20260923 20:51:12.749744 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:12.748919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:36123)
0923 20:51:12.749360 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:12.749441 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":240,"threads_started":1}
I20260923 20:51:12.781240 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:12.780288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:39041)
0923 20:51:12.780919 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:12.781016 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":331,"threads_started":1}
I20260923 20:51:13.269403 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:13.268454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:50815)
0923 20:51:13.269022 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:13.269179 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":269,"threads_started":1}
I20260923 20:51:13.751880 22553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:13.750969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:42201)
0923 20:51:13.751528 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:13.751629 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":283,"threads_started":1}
I20260923 20:51:13.783351 22554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:13.782366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:36719)
0923 20:51:13.782992 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:13.783117 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":350,"threads_started":1}
I20260923 20:51:14.205073 22555 raft_consensus.cc:482] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:14.205320 22555 raft_consensus.cc:504] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd4e89be4924cd58df6745904779990" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 35549 } } peers { permanent_uuid: "e44dec8a18b6439da47ceb3945af781a" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 35427 } } peers { permanent_uuid: "5e5a5e3f6cd64ceca18e4ace46f3b7e5" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 35435 } }
I20260923 20:51:14.206063 22555 leader_election.cc:288] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549), 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435)
W20260923 20:51:14.207295 22133 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e5a5e3f6cd64ceca18e4ace46f3b7e5 (127.5.71.131:35435): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
W20260923 20:51:14.207505 22135 leader_election.cc:341] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcd4e89be4924cd58df6745904779990 (127.5.71.130:35549): Not found: Tablet not found: 40fe556c05534286859e13ff2160ed4d
I20260923 20:51:14.207691 22135 leader_election.cc:302] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e44dec8a18b6439da47ceb3945af781a; no voters: 5e5a5e3f6cd64ceca18e4ace46f3b7e5, bcd4e89be4924cd58df6745904779990
I20260923 20:51:14.208005 22555 raft_consensus.cc:2726] T 40fe556c05534286859e13ff2160ed4d P e44dec8a18b6439da47ceb3945af781a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 20:51:14.271703 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:14.270547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:52363)
0923 20:51:14.271279 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:14.271433 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":363,"threads_started":1}
I20260923 20:51:14.753794 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:14.752878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:58935)
0923 20:51:14.753369 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:14.753463 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":253,"threads_started":1}
I20260923 20:51:14.785360 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:14.784463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:57485)
0923 20:51:14.785023 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:14.785127 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":302,"threads_started":1}
I20260923 20:51:15.273495 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:15.272777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:58695)
0923 20:51:15.273240 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:15.273332 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":225,"threads_started":1}
I20260923 20:51:15.755784 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:15.754910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:37609)
0923 20:51:15.755468 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:15.755565 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":294,"threads_started":1}
I20260923 20:51:15.787375 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:15.786500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:37193)
0923 20:51:15.787045 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:15.787184 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":294,"threads_started":1}
I20260923 20:51:16.275291 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:16.274440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:33699)
0923 20:51:16.274981 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:16.275069 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":261,"threads_started":1}
I20260923 20:51:16.757932 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:16.757035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:55447)
0923 20:51:16.757541 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:16.757644 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":262,"threads_started":1}
I20260923 20:51:16.789513 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:16.788789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.129:46599)
0923 20:51:16.789242 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:16.789332 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":233,"threads_started":1}
I20260923 20:51:17.277431 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:17.276500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.130:34469)
0923 20:51:17.277104 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:17.277258 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":312,"threads_started":1}
I20260923 20:51:17.733451 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22076
I20260923 20:51:17.750701 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:17.734651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.5.71.129:35427)
0923 20:51:17.735332 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:17.735345 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:17.735383 (+ 38us) server_negotiation.cc:407] Connection header received
0923 20:51:17.735487 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:17.735501 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:17.735874 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:17.736082 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:17.737920 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:17.738969 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:17.740470 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:17.741137 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:17.741298 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:17.744392 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:17.744407 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:17.744423 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:17.744461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:17.747447 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:17.748319 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:17.748327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:17.748329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:17.748494 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:17.748935 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:17.748983 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:17.748985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:17.749390 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:17.749742 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:17.750079 (+ 337us) server_negotiation.cc:299] Negotiation successful
0923 20:51:17.750379 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":693,"threads_started":1}
I20260923 20:51:17.752288 22243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:17.760231 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:17.759405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:45223 (local address 127.5.71.131:59799)
0923 20:51:17.759953 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:17.760079 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.190:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":259,"threads_started":1}
I20260923 20:51:17.848484 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22076
I20260923 20:51:17.867092 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22251
I20260923 20:51:17.882464 22570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:17.867983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49672 (local address 127.5.71.130:35549)
0923 20:51:17.868606 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:17.868611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:17.868627 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:51:17.868748 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:17.868752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:17.868813 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:17.868956 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:17.870413 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:17.871191 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:17.872529 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:17.872975 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:17.873110 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:17.876296 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:17.876302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:17.876305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:17.876345 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:17.879201 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:17.880094 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:17.880101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:17.880104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:17.880252 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:17.880686 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:17.880740 (+ 54us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:17.880744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:17.881082 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:17.881441 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:17.881779 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 20:51:17.882032 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":336,"threads_started":1}
I20260923 20:51:17.883313 22374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:17.972759 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22251
I20260923 20:51:17.991920 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22382
I20260923 20:51:18.008219 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:17.992860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39690 (local address 127.5.71.131:35435)
0923 20:51:17.993474 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:17.993479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:17.993499 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:51:17.993573 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:17.993578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:17.993669 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:17.993776 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:17.995332 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:17.996244 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:17.997694 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:17.998141 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:17.998258 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:18.001353 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:18.001359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:18.001363 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:18.001404 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:18.004637 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:18.005659 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:18.005663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:18.005665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:18.005902 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:18.006356 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:18.006361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:18.006363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:18.006781 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:18.007046 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:18.007408 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 20:51:18.007663 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":331,"threads_started":1}
I20260923 20:51:18.008993 22505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:18.100548 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22382
2026-09-23T20:51:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65462 ms)
[----------] 30 tests from SecurityITest (317873 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:18 dist-test-slave-dn29 krb5kdc[22576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:18 dist-test-slave-dn29 krb5kdc[22576](info): set up 2 sockets
Sep 23 20:51:18 dist-test-slave-dn29 krb5kdc[22576](info): commencing operation
krb5kdc: starting...
W20260923 20:51:19.297524 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.007s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:19 dist-test-slave-dn29 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:51:19.580973 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:41953
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:41953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:19.928610 22589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:19.928932 22589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:19.929098 22589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:19.937016 22589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:19.937143 22589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:19.937250 22589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:19.937362 22589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:19.937440 22589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:19.947778 22589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:41953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:41953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:19.949595 22589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:19.951333 22589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:19.964108 22594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:19.964161 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:19.964185 22595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:19.965235 22589 server_base.cc:1034] running on GCE node
I20260923 20:51:19.966110 22589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:19.967622 22589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:19.968822 22589 hybrid_clock.cc:648] HybridClock initialized: now 1790196679968767 us; error 41 us; skew 500 ppm
I20260923 20:51:19.969224 22589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:19 dist-test-slave-dn29 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196679, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:19.979481 22589 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:51:19.981385 22589 webserver.cc:460] Webserver started at http://127.5.71.190:43877/ using document root <none> and password file <none>
I20260923 20:51:19.982159 22589 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:19.982308 22589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:19.982666 22589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:19.985894 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "3b07b774529047239a62810e10014782"
format_stamp: "Formatted at 2026-09-23 20:51:19 on dist-test-slave-dn29"
server_key: "4b11ed8ae28b59d072948845d2cf26a4"
server_key_iv: "f5218694d4151b38a351316552b10989"
server_key_version: "encryptionkey@0"
I20260923 20:51:19.986706 22589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "3b07b774529047239a62810e10014782"
format_stamp: "Formatted at 2026-09-23 20:51:19 on dist-test-slave-dn29"
server_key: "4b11ed8ae28b59d072948845d2cf26a4"
server_key_iv: "f5218694d4151b38a351316552b10989"
server_key_version: "encryptionkey@0"
I20260923 20:51:19.992453 22589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:51:19.996912 22604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:19.998710 22589 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260923 20:51:19.998950 22589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "3b07b774529047239a62810e10014782"
format_stamp: "Formatted at 2026-09-23 20:51:19 on dist-test-slave-dn29"
server_key: "4b11ed8ae28b59d072948845d2cf26a4"
server_key_iv: "f5218694d4151b38a351316552b10989"
server_key_version: "encryptionkey@0"
I20260923 20:51:19.999178 22589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:20.019160 22589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:20.024153 22589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:20.024478 22589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:20.044701 22589 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:41953
I20260923 20:51:20.044713 22655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:41953 every 8 connection(s)
I20260923 20:51:20.046218 22589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:20.047868 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22589
I20260923 20:51:20.048148 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:20.049068 5406 external_mini_cluster.cc:1428] Setting key 613bc7a0c8a173fa58bea26ff8e50c8e
I20260923 20:51:20.051517 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 23 20:51:20 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:20.065374 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782: Bootstrap starting.
I20260923 20:51:20.069456 22656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:20.070780 22656 log.cc:824] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:20.074231 22656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782: No bootstrap required, opened a new log
I20260923 20:51:20.074875 22659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:20.051557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.5.71.190:41953)
0923 20:51:20.052457 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:20.052470 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:20.052514 (+ 44us) server_negotiation.cc:407] Connection header received
0923 20:51:20.054342 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:20.054366 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:20.054839 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:20.055298 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:20.057082 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:20.058592 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:20.060169 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:20.060999 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:20.061223 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:20.066493 (+ 5270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:20.066503 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:20.066522 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:20.066572 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:20.070466 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:20.071338 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:20.071346 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:20.071354 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:20.071553 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:20.072032 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:20.072036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:20.072038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:20.072602 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:20.072915 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:20.073363 (+ 448us) server_negotiation.cc:299] Negotiation successful
0923 20:51:20.073816 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":428,"threads_started":1}
I20260923 20:51:20.083333 22656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b07b774529047239a62810e10014782" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41953 } }
I20260923 20:51:20.083952 22656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:20.084132 22656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b07b774529047239a62810e10014782, State: Initialized, Role: FOLLOWER
I20260923 20:51:20.084934 22656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b07b774529047239a62810e10014782" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41953 } }
I20260923 20:51:20.085187 22656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:20.085379 22656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:20.085587 22656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:20.088214 22656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b07b774529047239a62810e10014782" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41953 } }
I20260923 20:51:20.088788 22656 leader_election.cc:302] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b07b774529047239a62810e10014782; no voters:
I20260923 20:51:20.089500 22656 leader_election.cc:288] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:20.089670 22661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:20.090492 22661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [term 1 LEADER]: Becoming Leader. State: Replica: 3b07b774529047239a62810e10014782, State: Running, Role: LEADER
I20260923 20:51:20.091209 22661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b07b774529047239a62810e10014782" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41953 } }
I20260923 20:51:20.092029 22656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:20.094532 22662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b07b774529047239a62810e10014782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b07b774529047239a62810e10014782" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41953 } } }
I20260923 20:51:20.095126 22662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:20.096336 22663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b07b774529047239a62810e10014782. Latest consensus state: current_term: 1 leader_uuid: "3b07b774529047239a62810e10014782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b07b774529047239a62810e10014782" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41953 } } }
I20260923 20:51:20.096755 22663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:20.097872 22669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:20.104008 22669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:20.114455 22669 catalog_manager.cc:1347] Generated new cluster ID: 5f9347bbcc2f477c85b76c499491cf3b
I20260923 20:51:20.114588 22669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:20.131691 22669 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:20.133956 22669 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:20.145262 22669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782: Generated new TSK 0
I20260923 20:51:20.146440 22669 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:51:20.224448 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41953
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:20.581362 22682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:20.581713 22682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:20.581871 22682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:20.590283 22682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:20.590464 22682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:20.590677 22682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:20.602386 22682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:20.604048 22682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:20.605513 22682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:20.621081 22688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:20.621183 22687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:20.621229 22690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:20.621877 22682 server_base.cc:1034] running on GCE node
I20260923 20:51:20.622717 22682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:20.623844 22682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:20.625150 22682 hybrid_clock.cc:648] HybridClock initialized: now 1790196680625087 us; error 54 us; skew 500 ppm
I20260923 20:51:20.625540 22682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:20 dist-test-slave-dn29 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196680, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:20.629159 22682 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:51:20.631004 22682 webserver.cc:460] Webserver started at http://127.5.71.129:44335/ using document root <none> and password file <none>
I20260923 20:51:20.631889 22682 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:20.632067 22682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:20.632441 22682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:20.635975 22682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "85492f7a445449088a109c2ab27280ef"
format_stamp: "Formatted at 2026-09-23 20:51:20 on dist-test-slave-dn29"
server_key: "9007e2b799baa9789cd04706a9e81c54"
server_key_iv: "6e06bf9333a6e1e8c98f1f2f880e16c3"
server_key_version: "encryptionkey@0"
I20260923 20:51:20.636966 22682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "85492f7a445449088a109c2ab27280ef"
format_stamp: "Formatted at 2026-09-23 20:51:20 on dist-test-slave-dn29"
server_key: "9007e2b799baa9789cd04706a9e81c54"
server_key_iv: "6e06bf9333a6e1e8c98f1f2f880e16c3"
server_key_version: "encryptionkey@0"
I20260923 20:51:20.643375 22682 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260923 20:51:20.647368 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:20.648947 22682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:20.649158 22682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "85492f7a445449088a109c2ab27280ef"
format_stamp: "Formatted at 2026-09-23 20:51:20 on dist-test-slave-dn29"
server_key: "9007e2b799baa9789cd04706a9e81c54"
server_key_iv: "6e06bf9333a6e1e8c98f1f2f880e16c3"
server_key_version: "encryptionkey@0"
I20260923 20:51:20.649381 22682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:20.672240 22682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:20.677371 22682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:20.677721 22682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:20.678874 22682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:20.680465 22682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:20.680588 22682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:20.680734 22682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:20.680815 22682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:20.718683 22682 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:41471
I20260923 20:51:20.718770 22809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:41471 every 8 connection(s)
I20260923 20:51:20.720089 22682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:20.724661 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22682
I20260923 20:51:20.724910 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:51:20.725584 5406 external_mini_cluster.cc:1428] Setting key ba2dc89db3908352b6fa6d2c83c2367e
Sep 23 20:51:20 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196680, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:20.741626 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:20.722851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:58475 (local address 127.5.71.190:41953)
0923 20:51:20.723290 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:20.723295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:20.724447 (+ 1152us) server_negotiation.cc:407] Connection header received
0923 20:51:20.726483 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:20.726489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:20.726586 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:20.726723 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:20.728359 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:20.729076 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:20.731273 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:20.731684 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:20.731833 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:20.735684 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:20.735690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:20.735693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:20.735729 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:20.738389 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:20.739158 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:20.739171 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:20.739175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:20.739305 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:20.739759 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:20.739764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:20.739767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:20.740093 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:20.740255 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:20.740999 (+ 744us) server_negotiation.cc:299] Negotiation successful
0923 20:51:20.741247 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":198,"threads_started":1}
I20260923 20:51:20.742722 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:20.723418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41953 (local address 127.5.71.129:58475)
0923 20:51:20.724284 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:20.724328 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:20.726173 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:20.726965 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:20.726973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:20.727522 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:20.728181 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:20.728199 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:20.729250 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:20.729260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:20.731105 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:20.731118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:20.731956 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:20.731963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:20.732199 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:20.732952 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:20.732976 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:20.735442 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:20.738540 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:20.738550 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:20.738568 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:20.739025 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:20.739424 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:20.739429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:20.739433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:20.739631 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:20.740226 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:20.740234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:20.740603 (+ 369us) client_negotiation.cc:769] Sending connection context
0923 20:51:20.740886 (+ 283us) client_negotiation.cc:240] Negotiation successful
0923 20:51:20.741451 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":428,"threads_started":1}
I20260923 20:51:20.744444 22810 heartbeater.cc:347] Connected to a master server at 127.5.71.190:41953
I20260923 20:51:20.744825 22810 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:20.745795 22810 heartbeater.cc:511] Master 127.5.71.190:41953 requested a full tablet report, sending...
I20260923 20:51:20.748699 22621 ts_manager.cc:194] Registered new tserver with Master: 85492f7a445449088a109c2ab27280ef (127.5.71.129:41471)
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:51:20.751147 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:58475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:51:20.785233 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41953
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:21.124966 22816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:21.125316 22816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:21.125533 22816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:21.133507 22816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:21.133670 22816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:21.133880 22816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:21.144644 22816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:21.146301 22816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:21.147781 22816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:21.162201 22824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:21.162271 22821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:21.162290 22822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:21.163025 22816 server_base.cc:1034] running on GCE node
I20260923 20:51:21.163841 22816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:21.164925 22816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:21.166193 22816 hybrid_clock.cc:648] HybridClock initialized: now 1790196681166143 us; error 58 us; skew 500 ppm
I20260923 20:51:21.166558 22816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:21 dist-test-slave-dn29 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196681, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:21.169982 22816 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:51:21.171651 22816 webserver.cc:460] Webserver started at http://127.5.71.130:40357/ using document root <none> and password file <none>
I20260923 20:51:21.172385 22816 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:21.172542 22816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:21.172870 22816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:21.175947 22816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "d8241da17dc84c339fbd4b59fbf51193"
format_stamp: "Formatted at 2026-09-23 20:51:21 on dist-test-slave-dn29"
server_key: "10cb585b9cd7756e6ebdc3da4285306a"
server_key_iv: "8d7649eb86c62b5d6775e9e5f15e385e"
server_key_version: "encryptionkey@0"
I20260923 20:51:21.176793 22816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "d8241da17dc84c339fbd4b59fbf51193"
format_stamp: "Formatted at 2026-09-23 20:51:21 on dist-test-slave-dn29"
server_key: "10cb585b9cd7756e6ebdc3da4285306a"
server_key_iv: "8d7649eb86c62b5d6775e9e5f15e385e"
server_key_version: "encryptionkey@0"
I20260923 20:51:21.182132 22816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260923 20:51:21.185940 22831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:21.187541 22816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260923 20:51:21.187736 22816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "d8241da17dc84c339fbd4b59fbf51193"
format_stamp: "Formatted at 2026-09-23 20:51:21 on dist-test-slave-dn29"
server_key: "10cb585b9cd7756e6ebdc3da4285306a"
server_key_iv: "8d7649eb86c62b5d6775e9e5f15e385e"
server_key_version: "encryptionkey@0"
I20260923 20:51:21.187994 22816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:21.219424 22816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:21.224433 22816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:21.224774 22816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:21.225885 22816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:21.227517 22816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:21.227596 22816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:21.227726 22816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:21.227809 22816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:21.266562 22816 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:43623
I20260923 20:51:21.266607 22943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:43623 every 8 connection(s)
I20260923 20:51:21.268028 22816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:21.272325 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22816
I20260923 20:51:21.272549 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:51:21.273138 5406 external_mini_cluster.cc:1428] Setting key 3ae17271b6fd5f444497e9f068af1a40
Sep 23 20:51:21 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196681, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:21.291429 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.270874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:41995 (local address 127.5.71.190:41953)
0923 20:51:21.271447 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:21.271454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:21.272607 (+ 1153us) server_negotiation.cc:407] Connection header received
0923 20:51:21.275218 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:21.275224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:21.275321 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:21.275459 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:21.277296 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.278149 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.280128 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.280636 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.280794 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.284701 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:21.284707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:21.284717 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:21.284756 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:21.288090 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:21.289004 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:21.289010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:21.289013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:21.289150 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:21.289607 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:21.289610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:21.289613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:21.289876 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:21.290054 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:21.290781 (+ 727us) server_negotiation.cc:299] Negotiation successful
0923 20:51:21.291027 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":239,"threads_started":1}
I20260923 20:51:21.292668 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.271485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41953 (local address 127.5.71.130:41995)
0923 20:51:21.272389 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:21.272460 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:21.274889 (+ 2429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:21.275742 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:21.275750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:21.276470 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:21.277103 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.277123 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.278330 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.278336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.279942 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.279981 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.280897 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.280905 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.281161 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.281923 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:21.281949 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:21.284420 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:21.288263 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:21.288273 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:21.288293 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:21.288826 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:21.289266 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:21.289271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:21.289274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:21.289472 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:21.290022 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:21.290032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:21.290411 (+ 379us) client_negotiation.cc:769] Sending connection context
0923 20:51:21.290690 (+ 279us) client_negotiation.cc:240] Negotiation successful
0923 20:51:21.291274 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":376,"threads_started":1}
I20260923 20:51:21.294488 22944 heartbeater.cc:347] Connected to a master server at 127.5.71.190:41953
I20260923 20:51:21.294901 22944 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:21.295928 22944 heartbeater.cc:511] Master 127.5.71.190:41953 requested a full tablet report, sending...
I20260923 20:51:21.297873 22621 ts_manager.cc:194] Registered new tserver with Master: d8241da17dc84c339fbd4b59fbf51193 (127.5.71.130:43623)
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:51:21.299114 22621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:41995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:51:21.332305 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41953
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:21.669690 22950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:21.670001 22950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:21.670100 22950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:21.677855 22950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:21.677968 22950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:21.678086 22950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:21.688002 22950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:41953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:21.689548 22950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:21.691021 22950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:21.705585 22955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:21.705699 22956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:21.706163 22958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:21.706995 22950 server_base.cc:1034] running on GCE node
I20260923 20:51:21.707653 22950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:21.708770 22950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:21.710021 22950 hybrid_clock.cc:648] HybridClock initialized: now 1790196681709984 us; error 28 us; skew 500 ppm
I20260923 20:51:21.710371 22950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:21 dist-test-slave-dn29 krb5kdc[22576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196681, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:21.713886 22950 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:51:21.715550 22950 webserver.cc:460] Webserver started at http://127.5.71.131:46583/ using document root <none> and password file <none>
I20260923 20:51:21.716348 22950 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:21.716490 22950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:21.716838 22950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:21.720153 22950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "594e97fc60e44e5ba4a9af1a7a025e8e"
format_stamp: "Formatted at 2026-09-23 20:51:21 on dist-test-slave-dn29"
server_key: "1698a64cafe8bb958c81d055bcd83469"
server_key_iv: "27818fe89d6369905aefafe655619a8e"
server_key_version: "encryptionkey@0"
I20260923 20:51:21.720995 22950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "594e97fc60e44e5ba4a9af1a7a025e8e"
format_stamp: "Formatted at 2026-09-23 20:51:21 on dist-test-slave-dn29"
server_key: "1698a64cafe8bb958c81d055bcd83469"
server_key_iv: "27818fe89d6369905aefafe655619a8e"
server_key_version: "encryptionkey@0"
I20260923 20:51:21.726449 22950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260923 20:51:21.730271 22965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:21.731909 22950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:21.732091 22950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "594e97fc60e44e5ba4a9af1a7a025e8e"
format_stamp: "Formatted at 2026-09-23 20:51:21 on dist-test-slave-dn29"
server_key: "1698a64cafe8bb958c81d055bcd83469"
server_key_iv: "27818fe89d6369905aefafe655619a8e"
server_key_version: "encryptionkey@0"
I20260923 20:51:21.732319 22950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:21.752429 22950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:21.755164 22810 heartbeater.cc:503] Master 127.5.71.190:41953 was elected leader, sending a full tablet report...
I20260923 20:51:21.758154 22950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:21.758419 22950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:21.759374 22950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:21.760829 22950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:21.760905 22950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:21.760990 22950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:21.761035 22950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:21.798156 22950 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:38989
I20260923 20:51:21.798206 23077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:38989 every 8 connection(s)
I20260923 20:51:21.799443 22950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:21.808256 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 22950
I20260923 20:51:21.808465 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:51:21.809046 5406 external_mini_cluster.cc:1428] Setting key 3cb28c6685c291bfa6abfa7f96f21e43
Sep 23 20:51:21 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196681, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:21.821234 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.802270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:34417 (local address 127.5.71.190:41953)
0923 20:51:21.802731 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:21.802737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:21.803928 (+ 1191us) server_negotiation.cc:407] Connection header received
0923 20:51:21.805890 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:21.805894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:21.805975 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:21.806079 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:21.807682 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.808728 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.810729 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.811239 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.811373 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.815379 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:21.815384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:21.815387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:21.815424 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:21.818159 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:21.818978 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:21.818982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:21.818984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:21.819104 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:21.819571 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:21.819575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:21.819577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:21.819919 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:21.820039 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:21.820739 (+ 700us) server_negotiation.cc:299] Negotiation successful
0923 20:51:21.820953 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":221,"threads_started":1}
I20260923 20:51:21.822301 23081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.802905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41953 (local address 127.5.71.131:34417)
0923 20:51:21.803672 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:21.803730 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:21.805569 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:21.806340 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:21.806348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:21.806934 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:21.807506 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.807523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.808892 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.808897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.810575 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.810590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.811502 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.811509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.811890 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.812804 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:21.812834 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:21.815153 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:21.818310 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:21.818317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:21.818332 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:21.818862 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:21.819238 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:21.819242 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:21.819244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:21.819447 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:21.820038 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:21.820044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:21.820400 (+ 356us) client_negotiation.cc:769] Sending connection context
0923 20:51:21.820654 (+ 254us) client_negotiation.cc:240] Negotiation successful
0923 20:51:21.821064 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":300,"threads_started":1}
I20260923 20:51:21.824141 23078 heartbeater.cc:347] Connected to a master server at 127.5.71.190:41953
I20260923 20:51:21.824462 23078 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:21.825347 23078 heartbeater.cc:511] Master 127.5.71.190:41953 requested a full tablet report, sending...
I20260923 20:51:21.826776 22620 ts_manager.cc:194] Registered new tserver with Master: 594e97fc60e44e5ba4a9af1a7a025e8e (127.5.71.131:38989)
I20260923 20:51:21.827670 22620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:34417
I20260923 20:51:21.838250 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:51:21.857127 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.843190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.5.71.190:41953)
0923 20:51:21.843409 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:21.843416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:21.843748 (+ 332us) server_negotiation.cc:407] Connection header received
0923 20:51:21.843908 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:21.843913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:21.843995 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:21.844126 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:21.845736 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.846507 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.847896 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.848316 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.848414 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.851057 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:21.851063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:21.851066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:21.851103 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:21.853971 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:21.854933 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:21.854938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:21.854940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:21.855084 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:21.855465 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:21.855469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:21.855471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:21.855918 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:21.856094 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:21.856503 (+ 409us) server_negotiation.cc:299] Negotiation successful
0923 20:51:21.856806 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 20:51:21.862093 22620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:51:21.869131 22620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:51:21.895831 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.883273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.5.71.131:38989)
0923 20:51:21.887009 (+ 3736us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:21.887018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:21.887047 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:51:21.887138 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:21.887143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:21.887292 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:21.887497 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:21.888209 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.889736 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.892509 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.894037 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.894179 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.894416 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:21.895325 (+ 909us) server_negotiation.cc:299] Negotiation successful
0923 20:51:21.895501 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3563,"thread_start_us":2610,"threads_started":1}
I20260923 20:51:21.895841 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.883202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:43623 (local address 127.0.0.1:36414)
0923 20:51:21.885778 (+ 2576us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:21.885798 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:21.885914 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:21.886900 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:21.886913 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:21.886955 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:21.887451 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.887473 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.889537 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.889544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.892234 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.892246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.894559 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.894571 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.894904 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.895012 (+ 108us) client_negotiation.cc:769] Sending connection context
0923 20:51:21.895146 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 20:51:21.895364 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2440,"mutex_wait_us":126,"thread_start_us":2368,"threads_started":1}
I20260923 20:51:21.896297 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.885957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47364)
0923 20:51:21.886473 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:21.886490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:21.886594 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:21.888672 (+ 2078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:21.888677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:21.888703 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:21.889048 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.889057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.891332 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.891338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.893484 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.893496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.895274 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.895282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.895450 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.895486 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 20:51:21.895605 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 20:51:21.895740 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":32,"thread_start_us":811,"threads_started":1}
I20260923 20:51:21.896615 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.883576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.5.71.130:43623)
0923 20:51:21.884469 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:21.884482 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:21.886040 (+ 1558us) server_negotiation.cc:407] Connection header received
0923 20:51:21.886153 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:21.886162 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:21.886431 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:21.886617 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:21.887739 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.889379 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.892395 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.894036 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.894181 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.894391 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:21.896112 (+ 1721us) server_negotiation.cc:299] Negotiation successful
0923 20:51:21.896290 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":515,"threads_started":1}
I20260923 20:51:21.896624 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.886552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47364 (local address 127.5.71.129:41471)
0923 20:51:21.888230 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:21.888236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:21.888255 (+ 19us) server_negotiation.cc:407] Connection header received
0923 20:51:21.888341 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:21.888346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:21.888487 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:21.888672 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:21.889226 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.891159 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.893635 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.895047 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.895201 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.895438 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:21.895658 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 20:51:21.895831 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1509,"thread_start_us":1340,"threads_started":1}
I20260923 20:51:21.897272 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:21.882998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38989 (local address 127.0.0.1:53398)
0923 20:51:21.885171 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:21.885231 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:21.885451 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:21.887480 (+ 2029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:21.887484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:21.887509 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:21.887937 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.887953 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.889949 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.889958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.892356 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:21.892371 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:21.894585 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:21.894591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:21.894888 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:21.895012 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 20:51:21.895146 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 20:51:21.896920 (+ 1774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1843,"mutex_wait_us":1714,"thread_start_us":1681,"threads_started":1}
I20260923 20:51:21.900308 22879 tablet_service.cc:1467] Processing CreateTablet for tablet 3a26fe4b82db4498a1e63580a11880ba (DEFAULT_TABLE table=test-table [id=0e50514821104bbeb89f6303f6d67040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:21.900509 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 3a26fe4b82db4498a1e63580a11880ba (DEFAULT_TABLE table=test-table [id=0e50514821104bbeb89f6303f6d67040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:21.900784 23013 tablet_service.cc:1467] Processing CreateTablet for tablet 3a26fe4b82db4498a1e63580a11880ba (DEFAULT_TABLE table=test-table [id=0e50514821104bbeb89f6303f6d67040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:21.901774 22879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a26fe4b82db4498a1e63580a11880ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:21.901940 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a26fe4b82db4498a1e63580a11880ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:21.902014 23013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a26fe4b82db4498a1e63580a11880ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:21.913558 23097 tablet_bootstrap.cc:492] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Bootstrap starting.
I20260923 20:51:21.916869 23098 tablet_bootstrap.cc:492] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Bootstrap starting.
I20260923 20:51:21.918067 23097 tablet_bootstrap.cc:654] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:21.918680 23099 tablet_bootstrap.cc:492] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Bootstrap starting.
I20260923 20:51:21.919623 23097 log.cc:824] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:21.920601 23098 tablet_bootstrap.cc:654] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:21.921891 23098 log.cc:824] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:21.921976 23099 tablet_bootstrap.cc:654] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:21.923023 23097 tablet_bootstrap.cc:492] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: No bootstrap required, opened a new log
I20260923 20:51:21.923161 23099 log.cc:824] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:21.923355 23097 ts_tablet_manager.cc:1397] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260923 20:51:21.925168 23098 tablet_bootstrap.cc:492] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: No bootstrap required, opened a new log
I20260923 20:51:21.925441 23098 ts_tablet_manager.cc:1397] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260923 20:51:21.926062 23099 tablet_bootstrap.cc:492] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: No bootstrap required, opened a new log
I20260923 20:51:21.926318 23099 ts_tablet_manager.cc:1397] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260923 20:51:21.933822 23099 raft_consensus.cc:348] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:21.934267 23099 raft_consensus.cc:374] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:21.934369 23099 raft_consensus.cc:729] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594e97fc60e44e5ba4a9af1a7a025e8e, State: Initialized, Role: FOLLOWER
I20260923 20:51:21.934511 23097 raft_consensus.cc:348] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:21.934973 23097 raft_consensus.cc:374] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:21.935076 23097 raft_consensus.cc:729] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8241da17dc84c339fbd4b59fbf51193, State: Initialized, Role: FOLLOWER
I20260923 20:51:21.935088 23099 consensus_queue.cc:260] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:21.935657 23097 consensus_queue.cc:260] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:21.936509 23078 heartbeater.cc:503] Master 127.5.71.190:41953 was elected leader, sending a full tablet report...
I20260923 20:51:21.936691 23098 raft_consensus.cc:348] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:21.937242 23099 ts_tablet_manager.cc:1428] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260923 20:51:21.937348 23098 raft_consensus.cc:374] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:21.937469 23098 raft_consensus.cc:729] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85492f7a445449088a109c2ab27280ef, State: Initialized, Role: FOLLOWER
I20260923 20:51:21.938374 23098 consensus_queue.cc:260] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:21.941831 23098 ts_tablet_manager.cc:1428] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260923 20:51:21.942145 23097 ts_tablet_manager.cc:1428] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Time spent starting tablet: real 0.019s user 0.019s sys 0.000s
I20260923 20:51:21.942649 22944 heartbeater.cc:503] Master 127.5.71.190:41953 was elected leader, sending a full tablet report...
W20260923 20:51:21.972947 22811 tablet.cc:2369] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:51:22.020332 22945 tablet.cc:2369] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:51:22.051337 23079 tablet.cc:2369] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:22.123731 23104 raft_consensus.cc:482] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:22.124158 23104 raft_consensus.cc:504] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:22.126024 23104 leader_election.cc:288] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 594e97fc60e44e5ba4a9af1a7a025e8e (127.5.71.131:38989), 85492f7a445449088a109c2ab27280ef (127.5.71.129:41471)
I20260923 20:51:22.134094 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.126619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:46079 (local address 127.5.71.129:41471)
0923 20:51:22.126854 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:22.126861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:22.127616 (+ 755us) server_negotiation.cc:407] Connection header received
0923 20:51:22.127897 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:22.127901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:22.127965 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:22.128064 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:22.128715 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.129861 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.131841 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.132955 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.133090 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.133157 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:22.133688 (+ 531us) server_negotiation.cc:299] Negotiation successful
0923 20:51:22.133833 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260923 20:51:22.134117 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.126619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.5.71.130:46079)
0923 20:51:22.127487 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.127507 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:22.127709 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:22.128081 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:22.128094 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:22.128124 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:22.128458 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:22.128467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.130002 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.130006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:22.131630 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:22.131642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.133283 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.133289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:22.133460 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.133525 (+ 65us) client_negotiation.cc:769] Sending connection context
0923 20:51:22.133635 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 20:51:22.133799 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":488,"threads_started":1}
I20260923 20:51:22.134625 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.126620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:54839 (local address 127.5.71.131:38989)
0923 20:51:22.126856 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:22.126863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:22.128000 (+ 1137us) server_negotiation.cc:407] Connection header received
0923 20:51:22.128225 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:22.128229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:22.128293 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:22.128411 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:22.129142 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.130080 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.132411 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.133464 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.133639 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.133707 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:22.134206 (+ 499us) server_negotiation.cc:299] Negotiation successful
0923 20:51:22.134348 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 20:51:22.134755 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.126618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:38989 (local address 127.5.71.130:54839)
0923 20:51:22.127871 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.127895 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:22.128054 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:22.128433 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:22.128441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:22.128468 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:22.128957 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:22.128972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.130241 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.130245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:22.132222 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:22.132240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.133758 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.133765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:22.133968 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.134000 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:51:22.134111 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 20:51:22.134254 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":256,"thread_start_us":535,"threads_started":1}
I20260923 20:51:22.135577 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26fe4b82db4498a1e63580a11880ba" candidate_uuid: "d8241da17dc84c339fbd4b59fbf51193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85492f7a445449088a109c2ab27280ef" is_pre_election: true
I20260923 20:51:22.135886 23033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26fe4b82db4498a1e63580a11880ba" candidate_uuid: "d8241da17dc84c339fbd4b59fbf51193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" is_pre_election: true
I20260923 20:51:22.136277 22765 raft_consensus.cc:2454] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8241da17dc84c339fbd4b59fbf51193 in term 0.
I20260923 20:51:22.136405 23033 raft_consensus.cc:2454] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8241da17dc84c339fbd4b59fbf51193 in term 0.
I20260923 20:51:22.137194 22834 leader_election.cc:302] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 594e97fc60e44e5ba4a9af1a7a025e8e, d8241da17dc84c339fbd4b59fbf51193; no voters:
I20260923 20:51:22.137655 23104 raft_consensus.cc:2781] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:51:22.137866 23104 raft_consensus.cc:482] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:51:22.138010 23104 raft_consensus.cc:3037] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:22.140842 23104 raft_consensus.cc:504] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:22.141532 23104 leader_election.cc:288] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [CANDIDATE]: Term 1 election: Requested vote from peers 594e97fc60e44e5ba4a9af1a7a025e8e (127.5.71.131:38989), 85492f7a445449088a109c2ab27280ef (127.5.71.129:41471)
I20260923 20:51:22.142199 23033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26fe4b82db4498a1e63580a11880ba" candidate_uuid: "d8241da17dc84c339fbd4b59fbf51193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e"
I20260923 20:51:22.142199 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26fe4b82db4498a1e63580a11880ba" candidate_uuid: "d8241da17dc84c339fbd4b59fbf51193" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85492f7a445449088a109c2ab27280ef"
I20260923 20:51:22.142452 23033 raft_consensus.cc:3037] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:22.142452 22765 raft_consensus.cc:3037] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:22.146198 23033 raft_consensus.cc:2454] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8241da17dc84c339fbd4b59fbf51193 in term 1.
I20260923 20:51:22.146199 22765 raft_consensus.cc:2454] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8241da17dc84c339fbd4b59fbf51193 in term 1.
I20260923 20:51:22.146824 22833 leader_election.cc:302] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85492f7a445449088a109c2ab27280ef, d8241da17dc84c339fbd4b59fbf51193; no voters:
I20260923 20:51:22.147244 23104 raft_consensus.cc:2781] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:22.147759 23104 raft_consensus.cc:686] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 1 LEADER]: Becoming Leader. State: Replica: d8241da17dc84c339fbd4b59fbf51193, State: Running, Role: LEADER
I20260923 20:51:22.148353 23104 consensus_queue.cc:237] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } }
I20260923 20:51:22.153337 22621 catalog_manager.cc:5495] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8241da17dc84c339fbd4b59fbf51193 (127.5.71.130). New cstate: current_term: 1 leader_uuid: "d8241da17dc84c339fbd4b59fbf51193" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8241da17dc84c339fbd4b59fbf51193" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 43623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 } health_report { overall_health: UNKNOWN } } }
I20260923 20:51:22.201211 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.195163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.5.71.130:43623)
0923 20:51:22.195483 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:22.195500 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:22.195571 (+ 71us) server_negotiation.cc:407] Connection header received
0923 20:51:22.195933 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:22.195938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:22.196031 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:22.196198 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:51:22.196594 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.197412 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.198836 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.199266 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.199407 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.200000 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:51:22.200553 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:51:22.200726 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:22.200813 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 20:51:22.200902 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260923 20:51:22.210259 22765 raft_consensus.cc:1261] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 1 FOLLOWER]: Refusing update from remote peer d8241da17dc84c339fbd4b59fbf51193: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:22.210258 23033 raft_consensus.cc:1261] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 1 FOLLOWER]: Refusing update from remote peer d8241da17dc84c339fbd4b59fbf51193: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:22.211367 23108 consensus_queue.cc:1035] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [LEADER]: Connected to new peer: Peer: permanent_uuid: "594e97fc60e44e5ba4a9af1a7a025e8e" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 38989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:22.211721 23104 consensus_queue.cc:1035] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85492f7a445449088a109c2ab27280ef" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 41471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:22.225351 23111 mvcc.cc:205] Tried to move back new op lower bound from 7332645610321653760 to 7332645610083807232. Current Snapshot: MvccSnapshot[applied={T|T < 7332645610321653760}]
I20260923 20:51:22.239738 22620 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:22.240123 22620 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:22.244014 22620 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b07b774529047239a62810e10014782: Sending DeleteTablet for 3 replicas of tablet 3a26fe4b82db4498a1e63580a11880ba
I20260923 20:51:22.245378 23013 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a26fe4b82db4498a1e63580a11880ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:22 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:53398
I20260923 20:51:22.245342 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a26fe4b82db4498a1e63580a11880ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:22 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:47364
I20260923 20:51:22.245565 22879 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a26fe4b82db4498a1e63580a11880ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:22 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:36414
I20260923 20:51:22.246256 23121 tablet_replica.cc:330] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: stopping tablet replica
I20260923 20:51:22.246752 23121 raft_consensus.cc:2229] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:22.247373 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22682
I20260923 20:51:22.247373 23121 raft_consensus.cc:2258] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:22.249836 23121 ts_tablet_manager.cc:1905] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:22.259635 23121 ts_tablet_manager.cc:1918] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:22.260020 23121 log.cc:1196] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/3a26fe4b82db4498a1e63580a11880ba
I20260923 20:51:22.260694 23121 ts_tablet_manager.cc:1939] T 3a26fe4b82db4498a1e63580a11880ba P 594e97fc60e44e5ba4a9af1a7a025e8e: Deleting consensus metadata
I20260923 20:51:22.262915 22607 catalog_manager.cc:4841] TS 594e97fc60e44e5ba4a9af1a7a025e8e (127.5.71.131:38989): tablet 3a26fe4b82db4498a1e63580a11880ba (table test-table [id=0e50514821104bbeb89f6303f6d67040]) successfully deleted
I20260923 20:51:22.264391 23122 tablet_replica.cc:330] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: stopping tablet replica
I20260923 20:51:22.264884 23122 raft_consensus.cc:2229] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:22.265394 23122 raft_consensus.cc:2258] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:22.265823 23123 tablet_replica.cc:330] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: stopping tablet replica
I20260923 20:51:22.266276 23123 raft_consensus.cc:2229] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:51:22.266963 23123 raft_consensus.cc:2258] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:22.270370 23123 ts_tablet_manager.cc:1905] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 23 20:51:22 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:51:22.277513 23123 ts_tablet_manager.cc:1918] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:22.277702 23123 log.cc:1196] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/3a26fe4b82db4498a1e63580a11880ba
I20260923 20:51:22.278312 23123 ts_tablet_manager.cc:1939] T 3a26fe4b82db4498a1e63580a11880ba P d8241da17dc84c339fbd4b59fbf51193: Deleting consensus metadata
I20260923 20:51:22.280315 22606 catalog_manager.cc:4841] TS d8241da17dc84c339fbd4b59fbf51193 (127.5.71.130:43623): tablet 3a26fe4b82db4498a1e63580a11880ba (table test-table [id=0e50514821104bbeb89f6303f6d67040]) successfully deleted
I20260923 20:51:22.281951 23122 ts_tablet_manager.cc:1905] T 3a26fe4b82db4498a1e63580a11880ba P 85492f7a445449088a109c2ab27280ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:22.284121 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.251013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47370 (local address 127.5.71.129:41471)
0923 20:51:22.251209 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:22.251217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:22.260466 (+ 9249us) server_negotiation.cc:407] Connection header received
0923 20:51:22.261035 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:22.261045 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:22.261151 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:22.261230 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:22.263291 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.264467 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.266773 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.267426 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.267573 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.273188 (+ 5615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:22.273207 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:22.273224 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:22.273274 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:22.278329 (+ 5055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:22.279511 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:22.279522 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:22.279525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:22.279717 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:22.281164 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:22.281169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:22.281172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:22.281707 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:22.282032 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:22.283408 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0923 20:51:22.283656 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:51:22.284897 22805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:22.385082 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22682
I20260923 20:51:22.404425 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22816
W20260923 20:51:22.404456 22606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:41471 (error 108)
I20260923 20:51:22.406997 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.406261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47376)
0923 20:51:22.406720 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.406830 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":253,"threads_started":1}
W20260923 20:51:22.407223 22606 catalog_manager.cc:4584] TS 85492f7a445449088a109c2ab27280ef (127.5.71.129:41471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a26fe4b82db4498a1e63580a11880ba: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Sep 23 20:51:22 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:51:22.425360 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.405404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.5.71.130:43623)
0923 20:51:22.405606 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:22.405614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:22.405723 (+ 109us) server_negotiation.cc:407] Connection header received
0923 20:51:22.405920 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:22.405926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:22.406025 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:22.406150 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:22.407963 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.408849 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.411342 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.412012 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.412127 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.417746 (+ 5619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:22.417763 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:22.417781 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:22.417825 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:22.421570 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:22.422520 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:22.422540 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:22.422543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:22.422736 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:22.423180 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:22.423186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:22.423197 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:22.423706 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:22.424160 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:22.424667 (+ 507us) server_negotiation.cc:299] Negotiation successful
0923 20:51:22.424938 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 20:51:22.426183 22939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:22.460963 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.460380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47392)
0923 20:51:22.460672 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.460791 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260923 20:51:22.462479 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.462000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47404)
0923 20:51:22.462224 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.462309 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260923 20:51:22.523418 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.522894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47416)
0923 20:51:22.523083 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.523185 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 20:51:22.525199 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.524558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47420)
0923 20:51:22.524829 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.524943 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 20:51:22.558270 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22816
I20260923 20:51:22.577694 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22950
Sep 23 20:51:22 dist-test-slave-dn29 krb5kdc[22576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:51:22.594343 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.578555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.5.71.131:38989)
0923 20:51:22.578761 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:22.578767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:22.578905 (+ 138us) server_negotiation.cc:407] Connection header received
0923 20:51:22.579141 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:22.579145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:22.579211 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:22.579332 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:22.580768 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.581520 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.582813 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:22.583211 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:22.583353 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:22.588337 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:22.588353 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:22.588371 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:22.588411 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:22.591260 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:22.592289 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:22.592297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:22.592300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:22.592471 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:22.592939 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:22.592943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:22.592945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:22.593303 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:22.593537 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:22.593896 (+ 359us) server_negotiation.cc:299] Negotiation successful
0923 20:51:22.594070 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:51:22.595029 23073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:22.606803 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.606234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47430)
0923 20:51:22.606505 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.606601 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260923 20:51:22.607971 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.607507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47442)
0923 20:51:22.607697 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.607780 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260923 20:51:22.686110 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22950
I20260923 20:51:22.708667 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 22589
I20260923 20:51:22.709966 22651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:22.830108 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:22.829605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:41471 (local address 127.0.0.1:47446)
0923 20:51:22.829791 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:22.829927 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260923 20:51:22.830116 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 22589
2026-09-23T20:51:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4734 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T20:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:22Z Disabled control of system clock
I20260923 20:51:22.914474 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:37955
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:37955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 20:51:23.264137 23134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:23.264453 23134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:23.264596 23134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:23.272176 23134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:23.272289 23134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:23.272351 23134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:23.272433 23134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:23.272496 23134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:23.282765 23134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:37955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:37955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:23.284193 23134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:23.286085 23134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:23.300325 23142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:23.300398 23140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:23.300303 23139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:23.301671 23134 server_base.cc:1034] running on GCE node
I20260923 20:51:23.302641 23134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:23.304273 23134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:23.305498 23134 hybrid_clock.cc:648] HybridClock initialized: now 1790196683305423 us; error 68 us; skew 500 ppm
I20260923 20:51:23.305859 23134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:23.315889 23134 webserver.cc:460] Webserver started at http://127.5.71.190:38041/ using document root <none> and password file <none>
I20260923 20:51:23.316663 23134 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:23.316769 23134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:23.317047 23134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:23.320253 23134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "e1aa81c5a7564d2db6200285d5e624ac"
format_stamp: "Formatted at 2026-09-23 20:51:23 on dist-test-slave-dn29"
server_key: "60a22e478b0a7b1d9e7f97a844a28ff5"
server_key_iv: "63bd159d2a3f46302042eaeeb682c396"
server_key_version: "encryptionkey@0"
I20260923 20:51:23.321084 23134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "e1aa81c5a7564d2db6200285d5e624ac"
format_stamp: "Formatted at 2026-09-23 20:51:23 on dist-test-slave-dn29"
server_key: "60a22e478b0a7b1d9e7f97a844a28ff5"
server_key_iv: "63bd159d2a3f46302042eaeeb682c396"
server_key_version: "encryptionkey@0"
I20260923 20:51:23.326759 23134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:51:23.330705 23148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:23.332494 23134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:23.332757 23134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "e1aa81c5a7564d2db6200285d5e624ac"
format_stamp: "Formatted at 2026-09-23 20:51:23 on dist-test-slave-dn29"
server_key: "60a22e478b0a7b1d9e7f97a844a28ff5"
server_key_iv: "63bd159d2a3f46302042eaeeb682c396"
server_key_version: "encryptionkey@0"
I20260923 20:51:23.332995 23134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:23.363843 23134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:23.364733 23134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:23.364988 23134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:23.385319 23134 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:37955
I20260923 20:51:23.385360 23199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:37955 every 8 connection(s)
I20260923 20:51:23.386744 23134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:23.390939 23200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:23.391992 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23134
I20260923 20:51:23.392227 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:23.393255 5406 external_mini_cluster.cc:1428] Setting key 4a88046da1205137b455bd826e88a5df
I20260923 20:51:23.411685 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac: Bootstrap starting.
I20260923 20:51:23.415583 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:23.396031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.5.71.190:37955)
0923 20:51:23.397594 (+ 1563us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:23.397610 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:23.397664 (+ 54us) server_negotiation.cc:407] Connection header received
0923 20:51:23.401936 (+ 4272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:23.401974 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:23.402716 (+ 742us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:23.404487 (+ 1771us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:23.405057 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:23.407537 (+ 2480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:23.409696 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:23.410441 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:23.410598 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:23.412223 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:23.412234 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:23.412255 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:23.412978 (+ 723us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:23.413083 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:23.413113 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:23.413723 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:23.413891 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 20:51:23.414285 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":432,"threads_started":1}
I20260923 20:51:23.416049 23200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:23.417187 23200 log.cc:824] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:23.420779 23200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac: No bootstrap required, opened a new log
I20260923 20:51:23.431504 23200 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aa81c5a7564d2db6200285d5e624ac" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37955 } }
I20260923 20:51:23.432063 23200 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:23.432168 23200 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1aa81c5a7564d2db6200285d5e624ac, State: Initialized, Role: FOLLOWER
I20260923 20:51:23.432967 23200 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aa81c5a7564d2db6200285d5e624ac" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37955 } }
I20260923 20:51:23.433291 23200 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:23.433426 23200 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:23.433602 23200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:23.437111 23200 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aa81c5a7564d2db6200285d5e624ac" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37955 } }
I20260923 20:51:23.437793 23200 leader_election.cc:302] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1aa81c5a7564d2db6200285d5e624ac; no voters:
I20260923 20:51:23.438920 23200 leader_election.cc:288] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:23.439141 23205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:23.440001 23205 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [term 1 LEADER]: Becoming Leader. State: Replica: e1aa81c5a7564d2db6200285d5e624ac, State: Running, Role: LEADER
I20260923 20:51:23.440685 23205 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aa81c5a7564d2db6200285d5e624ac" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37955 } }
I20260923 20:51:23.441324 23200 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:23.444546 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1aa81c5a7564d2db6200285d5e624ac. Latest consensus state: current_term: 1 leader_uuid: "e1aa81c5a7564d2db6200285d5e624ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aa81c5a7564d2db6200285d5e624ac" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37955 } } }
I20260923 20:51:23.444586 23206 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1aa81c5a7564d2db6200285d5e624ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1aa81c5a7564d2db6200285d5e624ac" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37955 } } }
I20260923 20:51:23.445012 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:23.445019 23206 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:23.453218 23214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:23.459777 23214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:23.466704 23223 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 20:51:23.471259 23214 catalog_manager.cc:1347] Generated new cluster ID: be2db2d533224b6a86ac816d07d7ed3a
I20260923 20:51:23.471441 23214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:23.492465 23214 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:23.494380 23214 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:23.506109 23214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac: Generated new TSK 0
I20260923 20:51:23.507050 23214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:23.512755 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37955
--builtin_ntp_servers=127.5.71.148:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 20:51:23.864591 23225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:23.864888 23225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:23.865026 23225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:23.872656 23225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:23.872759 23225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:23.872926 23225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:23.882555 23225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:23.883981 23225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:23.885805 23225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:23.899341 23230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:23.899341 23233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:23.899379 23231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:23.900720 23225 server_base.cc:1034] running on GCE node
I20260923 20:51:23.901377 23225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:23.902383 23225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:23.903731 23225 hybrid_clock.cc:648] HybridClock initialized: now 1790196683903650 us; error 68 us; skew 500 ppm
I20260923 20:51:23.904085 23225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:23.906656 23225 webserver.cc:460] Webserver started at http://127.5.71.129:34645/ using document root <none> and password file <none>
I20260923 20:51:23.907373 23225 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:23.907500 23225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:23.907907 23225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:23.910902 23225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "80fa979d84e84f21a8fac72495334c95"
format_stamp: "Formatted at 2026-09-23 20:51:23 on dist-test-slave-dn29"
server_key: "2a993bd9ac77072dbdb1486833cae745"
server_key_iv: "2cd70532c4ae72012277425e92f93565"
server_key_version: "encryptionkey@0"
I20260923 20:51:23.911840 23225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "80fa979d84e84f21a8fac72495334c95"
format_stamp: "Formatted at 2026-09-23 20:51:23 on dist-test-slave-dn29"
server_key: "2a993bd9ac77072dbdb1486833cae745"
server_key_iv: "2cd70532c4ae72012277425e92f93565"
server_key_version: "encryptionkey@0"
I20260923 20:51:23.917369 23225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260923 20:51:23.921428 23239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:23.923050 23225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:23.923250 23225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "80fa979d84e84f21a8fac72495334c95"
format_stamp: "Formatted at 2026-09-23 20:51:23 on dist-test-slave-dn29"
server_key: "2a993bd9ac77072dbdb1486833cae745"
server_key_iv: "2cd70532c4ae72012277425e92f93565"
server_key_version: "encryptionkey@0"
I20260923 20:51:23.923489 23225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:23.958482 23225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:23.959383 23225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:23.959708 23225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:23.960843 23225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:23.962441 23225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:23.962566 23225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:23.962702 23225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:23.962786 23225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:23.999336 23225 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:44299
I20260923 20:51:23.999429 23351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:44299 every 8 connection(s)
I20260923 20:51:24.000813 23225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:24.002564 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23225
I20260923 20:51:24.002753 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:51:24.003229 5406 external_mini_cluster.cc:1428] Setting key 00b311f3865d2d07979b624219e0cd6f
I20260923 20:51:24.007419 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37955
--builtin_ntp_servers=127.5.71.148:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 20:51:24.016929 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.003918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:48981 (local address 127.5.71.190:37955)
0923 20:51:24.004432 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:24.004436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:24.006854 (+ 2418us) server_negotiation.cc:407] Connection header received
0923 20:51:24.008532 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:24.008541 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:24.008666 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:24.008796 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:24.010035 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.011174 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.013118 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.014001 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.014215 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:24.015961 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:24.015966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:24.015970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:24.015991 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:24.016036 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:24.016049 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:24.016164 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:24.016448 (+ 284us) server_negotiation.cc:299] Negotiation successful
0923 20:51:24.016540 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
I20260923 20:51:24.018038 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.004617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:37955 (local address 127.5.71.129:48981)
0923 20:51:24.005813 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:24.005878 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:24.008191 (+ 2313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:24.009038 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:24.009047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:24.009158 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:24.009833 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:24.009864 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.011384 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.011388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:24.012880 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:24.012898 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.014280 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.014286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:24.014516 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:24.015529 (+ 1013us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:24.015535 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:24.015599 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:24.015724 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:24.016169 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:24.016183 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:24.016194 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 20:51:24.016400 (+ 206us) client_negotiation.cc:240] Negotiation successful
0923 20:51:24.016870 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":1227,"threads_started":1}
I20260923 20:51:24.019995 23352 heartbeater.cc:347] Connected to a master server at 127.5.71.190:37955
I20260923 20:51:24.020360 23352 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:24.021219 23352 heartbeater.cc:511] Master 127.5.71.190:37955 requested a full tablet report, sending...
I20260923 20:51:24.023878 23165 ts_manager.cc:194] Registered new tserver with Master: 80fa979d84e84f21a8fac72495334c95 (127.5.71.129:44299)
I20260923 20:51:24.026458 23165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.129:48981
W20260923 20:51:24.362471 23356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:24.362753 23356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:24.362839 23356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:24.370290 23356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:24.370404 23356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:24.370514 23356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:24.380151 23356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:24.381412 23356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:24.383193 23356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:24.398845 23364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:24.398866 23361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:24.399255 23362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:24.400134 23356 server_base.cc:1034] running on GCE node
I20260923 20:51:24.400846 23356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:24.402048 23356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:24.403306 23356 hybrid_clock.cc:648] HybridClock initialized: now 1790196684403231 us; error 61 us; skew 500 ppm
I20260923 20:51:24.403673 23356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:24.406558 23356 webserver.cc:460] Webserver started at http://127.5.71.130:41073/ using document root <none> and password file <none>
I20260923 20:51:24.407238 23356 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:24.407328 23356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:24.407665 23356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:24.410626 23356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "802fb6b8314a44bc80f9610f20bfd350"
format_stamp: "Formatted at 2026-09-23 20:51:24 on dist-test-slave-dn29"
server_key: "44df6c2bc0a3c8d00dfe21482d11dcc7"
server_key_iv: "237fddfd6e798005914f30a6edd18640"
server_key_version: "encryptionkey@0"
I20260923 20:51:24.411466 23356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "802fb6b8314a44bc80f9610f20bfd350"
format_stamp: "Formatted at 2026-09-23 20:51:24 on dist-test-slave-dn29"
server_key: "44df6c2bc0a3c8d00dfe21482d11dcc7"
server_key_iv: "237fddfd6e798005914f30a6edd18640"
server_key_version: "encryptionkey@0"
I20260923 20:51:24.416859 23356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260923 20:51:24.420681 23370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:24.422103 23356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:24.422302 23356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "802fb6b8314a44bc80f9610f20bfd350"
format_stamp: "Formatted at 2026-09-23 20:51:24 on dist-test-slave-dn29"
server_key: "44df6c2bc0a3c8d00dfe21482d11dcc7"
server_key_iv: "237fddfd6e798005914f30a6edd18640"
server_key_version: "encryptionkey@0"
I20260923 20:51:24.422540 23356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:24.439059 23356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:24.440128 23356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:24.440482 23356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:24.441653 23356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:24.443482 23356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:24.443645 23356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:24.443827 23356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:24.443943 23356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260923 20:51:24.482641 23356 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:46291
I20260923 20:51:24.482690 23482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:46291 every 8 connection(s)
I20260923 20:51:24.484118 23356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:24.489148 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23356
I20260923 20:51:24.489346 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:51:24.489775 5406 external_mini_cluster.cc:1428] Setting key 6ef54601ea89e2fa27d40b62073bf6ed
I20260923 20:51:24.492978 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37955
--builtin_ntp_servers=127.5.71.148:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 20:51:24.498327 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.487176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:41907 (local address 127.5.71.190:37955)
0923 20:51:24.487387 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:24.487392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:24.488941 (+ 1549us) server_negotiation.cc:407] Connection header received
0923 20:51:24.491151 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:24.491157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:24.491231 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:24.491363 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:24.492820 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.493639 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.495290 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.495710 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.495869 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:24.497356 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:24.497361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:24.497364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:24.497384 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:24.497417 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:24.497429 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:24.497539 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:24.497900 (+ 361us) server_negotiation.cc:299] Negotiation successful
0923 20:51:24.498004 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 20:51:24.499452 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.487664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:37955 (local address 127.5.71.130:41907)
0923 20:51:24.488766 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:24.488814 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:24.490873 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:24.491759 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:24.491775 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:24.491941 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:24.492621 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:24.492640 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.493808 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.493818 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:24.495131 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:24.495145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.495988 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.495998 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:24.496227 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:24.496977 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:24.496979 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:24.497023 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:24.497129 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:24.497650 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:24.497661 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:24.497668 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 20:51:24.497869 (+ 201us) client_negotiation.cc:240] Negotiation successful
0923 20:51:24.498304 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":413,"threads_started":1}
I20260923 20:51:24.501138 23483 heartbeater.cc:347] Connected to a master server at 127.5.71.190:37955
I20260923 20:51:24.501524 23483 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:24.502259 23483 heartbeater.cc:511] Master 127.5.71.190:37955 requested a full tablet report, sending...
I20260923 20:51:24.503749 23165 ts_manager.cc:194] Registered new tserver with Master: 802fb6b8314a44bc80f9610f20bfd350 (127.5.71.130:46291)
I20260923 20:51:24.504637 23165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.130:41907
W20260923 20:51:24.833767 23486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:24.834053 23486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:24.834221 23486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:24.841825 23486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:24.841933 23486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:24.842098 23486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:24.851663 23486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:37955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:24.853230 23486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:24.855134 23486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:24.869918 23494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:24.870016 23492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:24.870460 23491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:24.873736 23486 server_base.cc:1034] running on GCE node
I20260923 20:51:24.874418 23486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:24.875432 23486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:24.876698 23486 hybrid_clock.cc:648] HybridClock initialized: now 1790196684876627 us; error 59 us; skew 500 ppm
I20260923 20:51:24.877179 23486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:24.880069 23486 webserver.cc:460] Webserver started at http://127.5.71.131:37309/ using document root <none> and password file <none>
I20260923 20:51:24.881048 23486 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:24.881187 23486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:24.881575 23486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:24.884812 23486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "6ad9512e562e46feb5f103ef42875f2d"
format_stamp: "Formatted at 2026-09-23 20:51:24 on dist-test-slave-dn29"
server_key: "8b6946614d456c2b460ac7016fd05f93"
server_key_iv: "735573797165bc263b2cf64f4c76a0ac"
server_key_version: "encryptionkey@0"
I20260923 20:51:24.885691 23486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "6ad9512e562e46feb5f103ef42875f2d"
format_stamp: "Formatted at 2026-09-23 20:51:24 on dist-test-slave-dn29"
server_key: "8b6946614d456c2b460ac7016fd05f93"
server_key_iv: "735573797165bc263b2cf64f4c76a0ac"
server_key_version: "encryptionkey@0"
I20260923 20:51:24.891736 23486 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260923 20:51:24.896116 23500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:24.897789 23486 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:51:24.897941 23486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "6ad9512e562e46feb5f103ef42875f2d"
format_stamp: "Formatted at 2026-09-23 20:51:24 on dist-test-slave-dn29"
server_key: "8b6946614d456c2b460ac7016fd05f93"
server_key_iv: "735573797165bc263b2cf64f4c76a0ac"
server_key_version: "encryptionkey@0"
I20260923 20:51:24.898159 23486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:24.915669 23486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:24.916635 23486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:24.916931 23486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:24.917990 23486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:24.919539 23486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:24.919639 23486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:24.919731 23486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:24.919886 23486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260923 20:51:24.958856 23486 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:44369
I20260923 20:51:24.958923 23612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:44369 every 8 connection(s)
I20260923 20:51:24.960358 23486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:24.961788 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23486
I20260923 20:51:24.961992 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:51:24.962539 5406 external_mini_cluster.cc:1428] Setting key a1436c4b676f46016c20ed2b45fa75b9
I20260923 20:51:24.974828 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.963458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:38209 (local address 127.5.71.190:37955)
0923 20:51:24.963658 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:24.963671 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:24.965537 (+ 1866us) server_negotiation.cc:407] Connection header received
0923 20:51:24.967321 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:24.967326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:24.967393 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:24.967478 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:24.968737 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.969606 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.971958 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.972439 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.972564 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:24.974055 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:24.974061 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:24.974064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:24.974087 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:24.974122 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:24.974135 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:24.974211 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:24.974447 (+ 236us) server_negotiation.cc:299] Negotiation successful
0923 20:51:24.974538 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:51:24.976023 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.964181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:37955 (local address 127.5.71.131:38209)
0923 20:51:24.965374 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:24.965421 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:24.967072 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:24.967691 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:24.967699 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:24.967932 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:24.968548 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:24.968567 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.969771 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.969775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:24.971728 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:24.971741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.972656 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:24.972662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:24.972950 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:24.973664 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:24.973667 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:24.973711 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:24.973832 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:24.974223 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:24.974231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:24.974237 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 20:51:24.974406 (+ 169us) client_negotiation.cc:240] Negotiation successful
0923 20:51:24.974729 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":407,"threads_started":1}
I20260923 20:51:24.977895 23613 heartbeater.cc:347] Connected to a master server at 127.5.71.190:37955
I20260923 20:51:24.978281 23613 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:24.979231 23613 heartbeater.cc:511] Master 127.5.71.190:37955 requested a full tablet report, sending...
I20260923 20:51:24.980829 23164 ts_manager.cc:194] Registered new tserver with Master: 6ad9512e562e46feb5f103ef42875f2d (127.5.71.131:44369)
I20260923 20:51:24.981842 23164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.131:38209
I20260923 20:51:24.992008 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:51:25.013649 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:24.996956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.5.71.190:37955)
0923 20:51:24.997257 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:24.997266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:24.997615 (+ 349us) server_negotiation.cc:407] Connection header received
0923 20:51:24.997879 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:24.997883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:24.997969 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:24.998091 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:24.998639 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:24.999524 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.001287 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.001792 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.002078 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.012793 (+ 10715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:25.012800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:25.012803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:25.012827 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:25.012867 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:25.012881 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:25.013001 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.013189 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.013302 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260923 20:51:25.018520 23164 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:51:25.026428 23164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:51:25.030407 23352 heartbeater.cc:503] Master 127.5.71.190:37955 was elected leader, sending a full tablet report...
I20260923 20:51:25.053138 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.041338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:44369 (local address 127.0.0.1:44262)
0923 20:51:25.042328 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.042368 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:25.042550 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:25.045225 (+ 2675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:25.045234 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:25.045264 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:25.045687 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.045703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.047778 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.047784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.050152 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.050164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.051995 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.052002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.052210 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.052268 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 20:51:25.052413 (+ 145us) client_negotiation.cc:240] Negotiation successful
0923 20:51:25.052672 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":140,"thread_start_us":509,"threads_started":1}
I20260923 20:51:25.053843 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.042865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33022)
0923 20:51:25.044985 (+ 2120us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.045001 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:25.045108 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:25.045906 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:25.045910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:25.045950 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:25.046619 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.046632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.048064 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.048076 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.050640 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.050661 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.052930 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.052937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.053162 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.053206 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 20:51:25.053303 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 20:51:25.053430 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2004,"mutex_wait_us":240,"spinlock_wait_cycles":4096,"thread_start_us":1539,"threads_started":1}
I20260923 20:51:25.053848 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.042390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.5.71.130:46291)
0923 20:51:25.043325 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.043337 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.044889 (+ 1552us) server_negotiation.cc:407] Connection header received
0923 20:51:25.045208 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.045216 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.045440 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.045612 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:25.046750 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.048254 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.050988 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.052231 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.052381 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.052663 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.053291 (+ 628us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.053464 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":625,"threads_started":1}
I20260923 20:51:25.054369 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.041732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.5.71.131:44369)
0923 20:51:25.044447 (+ 2715us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.044459 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.044503 (+ 44us) server_negotiation.cc:407] Connection header received
0923 20:51:25.044639 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.044652 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.044952 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.045910 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:25.046137 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.047621 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.050363 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.051777 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.051946 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.052301 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.052477 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.054009 (+ 1532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2490,"thread_start_us":1798,"threads_started":1}
I20260923 20:51:25.055681 23630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.043531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.5.71.129:44299)
0923 20:51:25.044214 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.044237 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.045225 (+ 988us) server_negotiation.cc:407] Connection header received
0923 20:51:25.045318 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.045324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.045514 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.045672 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:25.046810 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.047926 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.050871 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.052405 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.052538 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.052791 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.054268 (+ 1477us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.054456 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":358,"threads_started":1}
I20260923 20:51:25.054404 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.042016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:46291 (local address 127.0.0.1:48134)
0923 20:51:25.044750 (+ 2734us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.044774 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:25.045010 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:25.045902 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:25.045907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:25.045948 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:25.046541 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.046561 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.048451 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.048461 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.050741 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.050764 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.052791 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.052797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.053037 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.053094 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 20:51:25.054040 (+ 946us) client_negotiation.cc:240] Negotiation successful
0923 20:51:25.054157 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2571,"mutex_wait_us":702,"spinlock_wait_cycles":7808,"thread_start_us":2065,"threads_started":1}
I20260923 20:51:25.058528 23548 tablet_service.cc:1467] Processing CreateTablet for tablet e499dd7e01a545c5a9638269090dc97e (DEFAULT_TABLE table=test-table [id=9dbdd4b546614522bb5501da16f2a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:25.060235 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e499dd7e01a545c5a9638269090dc97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:25.059873 23287 tablet_service.cc:1467] Processing CreateTablet for tablet e499dd7e01a545c5a9638269090dc97e (DEFAULT_TABLE table=test-table [id=9dbdd4b546614522bb5501da16f2a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:25.061347 23287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e499dd7e01a545c5a9638269090dc97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:25.062052 23418 tablet_service.cc:1467] Processing CreateTablet for tablet e499dd7e01a545c5a9638269090dc97e (DEFAULT_TABLE table=test-table [id=9dbdd4b546614522bb5501da16f2a703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:25.063118 23418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e499dd7e01a545c5a9638269090dc97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:25.075165 23631 tablet_bootstrap.cc:492] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Bootstrap starting.
I20260923 20:51:25.077530 23633 tablet_bootstrap.cc:492] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Bootstrap starting.
I20260923 20:51:25.079407 23631 tablet_bootstrap.cc:654] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:25.081501 23633 tablet_bootstrap.cc:654] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:25.081452 23631 log.cc:824] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:25.082839 23632 tablet_bootstrap.cc:492] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Bootstrap starting.
I20260923 20:51:25.082984 23633 log.cc:824] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:25.085573 23631 tablet_bootstrap.cc:492] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: No bootstrap required, opened a new log
I20260923 20:51:25.085891 23631 ts_tablet_manager.cc:1397] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260923 20:51:25.086355 23633 tablet_bootstrap.cc:492] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: No bootstrap required, opened a new log
I20260923 20:51:25.086637 23633 ts_tablet_manager.cc:1397] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260923 20:51:25.087519 23632 tablet_bootstrap.cc:654] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:25.088912 23632 log.cc:824] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:25.093153 23632 tablet_bootstrap.cc:492] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: No bootstrap required, opened a new log
I20260923 20:51:25.093492 23632 ts_tablet_manager.cc:1397] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260923 20:51:25.094933 23631 raft_consensus.cc:348] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.095453 23631 raft_consensus.cc:374] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:25.095556 23631 raft_consensus.cc:729] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802fb6b8314a44bc80f9610f20bfd350, State: Initialized, Role: FOLLOWER
I20260923 20:51:25.096278 23631 consensus_queue.cc:260] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.097496 23483 heartbeater.cc:503] Master 127.5.71.190:37955 was elected leader, sending a full tablet report...
I20260923 20:51:25.097576 23633 raft_consensus.cc:348] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.098062 23633 raft_consensus.cc:374] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:25.098163 23633 raft_consensus.cc:729] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fa979d84e84f21a8fac72495334c95, State: Initialized, Role: FOLLOWER
I20260923 20:51:25.098795 23633 consensus_queue.cc:260] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.099606 23631 ts_tablet_manager.cc:1428] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260923 20:51:25.102419 23633 ts_tablet_manager.cc:1428] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260923 20:51:25.107344 23632 raft_consensus.cc:348] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.107724 23632 raft_consensus.cc:374] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:25.107884 23632 raft_consensus.cc:729] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad9512e562e46feb5f103ef42875f2d, State: Initialized, Role: FOLLOWER
I20260923 20:51:25.108469 23632 consensus_queue.cc:260] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.109560 23613 heartbeater.cc:503] Master 127.5.71.190:37955 was elected leader, sending a full tablet report...
I20260923 20:51:25.110214 23632 ts_tablet_manager.cc:1428] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Time spent starting tablet: real 0.017s user 0.009s sys 0.004s
I20260923 20:51:25.190090 23639 raft_consensus.cc:482] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:25.190459 23639 raft_consensus.cc:504] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.192574 23639 leader_election.cc:288] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80fa979d84e84f21a8fac72495334c95 (127.5.71.129:44299), 802fb6b8314a44bc80f9610f20bfd350 (127.5.71.130:46291)
I20260923 20:51:25.199666 23615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.192826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.5.71.131:41925)
0923 20:51:25.193119 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.193144 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:25.193321 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:25.193872 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:25.193881 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:25.193920 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:25.194117 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.194126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.195360 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.195367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.197768 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.197783 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.199062 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.199067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.199198 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.199225 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 20:51:25.199303 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 20:51:25.199408 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260923 20:51:25.199733 23630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.193124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:41925 (local address 127.5.71.129:44299)
0923 20:51:25.193334 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.193342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.193369 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:51:25.193511 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.193519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.193619 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.193733 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:25.194282 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.195167 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.197952 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.198909 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.199068 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.199155 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.199348 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.199477 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 20:51:25.200850 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.193084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:46291 (local address 127.5.71.131:48945)
0923 20:51:25.194341 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.194358 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:25.194517 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:25.194929 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:25.194933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:25.194958 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:25.195398 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.195410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.196610 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.196616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.198526 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:25.198537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.200123 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.200131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:25.200335 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.200384 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 20:51:25.200497 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 20:51:25.200607 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1157,"thread_start_us":525,"threads_started":1}
I20260923 20:51:25.200932 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.193272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:48945 (local address 127.5.71.130:46291)
0923 20:51:25.193576 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.193584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.194482 (+ 898us) server_negotiation.cc:407] Connection header received
0923 20:51:25.194686 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.194690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.194792 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.194901 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:25.195598 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.196439 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.198719 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.199947 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.200085 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.200144 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.200547 (+ 403us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.200688 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260923 20:51:25.201138 23307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e499dd7e01a545c5a9638269090dc97e" candidate_uuid: "6ad9512e562e46feb5f103ef42875f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80fa979d84e84f21a8fac72495334c95" is_pre_election: true
I20260923 20:51:25.201771 23307 raft_consensus.cc:2454] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9512e562e46feb5f103ef42875f2d in term 0.
I20260923 20:51:25.202473 23504 leader_election.cc:302] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9512e562e46feb5f103ef42875f2d, 80fa979d84e84f21a8fac72495334c95; no voters:
I20260923 20:51:25.202402 23438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e499dd7e01a545c5a9638269090dc97e" candidate_uuid: "6ad9512e562e46feb5f103ef42875f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802fb6b8314a44bc80f9610f20bfd350" is_pre_election: true
I20260923 20:51:25.202859 23639 raft_consensus.cc:2781] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:51:25.203006 23639 raft_consensus.cc:482] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:51:25.203120 23639 raft_consensus.cc:3037] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:25.203106 23438 raft_consensus.cc:2454] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad9512e562e46feb5f103ef42875f2d in term 0.
I20260923 20:51:25.205571 23639 raft_consensus.cc:504] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
I20260923 20:51:25.206247 23639 leader_election.cc:288] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [CANDIDATE]: Term 1 election: Requested vote from peers 80fa979d84e84f21a8fac72495334c95 (127.5.71.129:44299), 802fb6b8314a44bc80f9610f20bfd350 (127.5.71.130:46291)
I20260923 20:51:25.206727 23307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e499dd7e01a545c5a9638269090dc97e" candidate_uuid: "6ad9512e562e46feb5f103ef42875f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80fa979d84e84f21a8fac72495334c95"
I20260923 20:51:25.206934 23307 raft_consensus.cc:3037] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:25.206853 23438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e499dd7e01a545c5a9638269090dc97e" candidate_uuid: "6ad9512e562e46feb5f103ef42875f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "802fb6b8314a44bc80f9610f20bfd350"
I20260923 20:51:25.207134 23438 raft_consensus.cc:3037] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:25.209299 23307 raft_consensus.cc:2454] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9512e562e46feb5f103ef42875f2d in term 1.
I20260923 20:51:25.209712 23504 leader_election.cc:302] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad9512e562e46feb5f103ef42875f2d, 80fa979d84e84f21a8fac72495334c95; no voters:
I20260923 20:51:25.209812 23438 raft_consensus.cc:2454] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad9512e562e46feb5f103ef42875f2d in term 1.
I20260923 20:51:25.210067 23639 raft_consensus.cc:2781] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:25.210573 23639 raft_consensus.cc:686] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 1 LEADER]: Becoming Leader. State: Replica: 6ad9512e562e46feb5f103ef42875f2d, State: Running, Role: LEADER
I20260923 20:51:25.211067 23639 consensus_queue.cc:237] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } }
W20260923 20:51:25.212318 23614 tablet.cc:2369] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:25.216295 23165 catalog_manager.cc:5495] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad9512e562e46feb5f103ef42875f2d (127.5.71.131). New cstate: current_term: 1 leader_uuid: "6ad9512e562e46feb5f103ef42875f2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ad9512e562e46feb5f103ef42875f2d" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 44369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 } health_report { overall_health: UNKNOWN } } }
W20260923 20:51:25.236428 23484 tablet.cc:2369] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:25.247478 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.240699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44276 (local address 127.5.71.131:44369)
0923 20:51:25.240919 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.240929 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.241059 (+ 130us) server_negotiation.cc:407] Connection header received
0923 20:51:25.241366 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.241371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.241461 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.241654 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:51:25.242142 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.243059 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.244838 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.245351 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.245517 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.246170 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:51:25.246770 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:51:25.246972 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.247059 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.247160 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260923 20:51:25.253413 23353 tablet.cc:2369] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:25.256906 23438 raft_consensus.cc:1261] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9512e562e46feb5f103ef42875f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:25.256932 23307 raft_consensus.cc:1261] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 1 FOLLOWER]: Refusing update from remote peer 6ad9512e562e46feb5f103ef42875f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:25.257961 23641 consensus_queue.cc:1035] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "802fb6b8314a44bc80f9610f20bfd350" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:25.258467 23639 consensus_queue.cc:1035] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80fa979d84e84f21a8fac72495334c95" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 44299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:25.272518 23646 mvcc.cc:205] Tried to move back new op lower bound from 7332645622801289216 to 7332645622629072896. Current Snapshot: MvccSnapshot[applied={T|T < 7332645622801289216}]
I20260923 20:51:25.273823 23644 mvcc.cc:205] Tried to move back new op lower bound from 7332645622801289216 to 7332645622629072896. Current Snapshot: MvccSnapshot[applied={T|T < 7332645622801289216}]
I20260923 20:51:25.289347 23164 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:25.289691 23164 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:25.293901 23164 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1aa81c5a7564d2db6200285d5e624ac: Sending DeleteTablet for 3 replicas of tablet e499dd7e01a545c5a9638269090dc97e
I20260923 20:51:25.295344 23548 tablet_service.cc:1514] Processing DeleteTablet for tablet e499dd7e01a545c5a9638269090dc97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:25 UTC) from {username='slave'} at 127.0.0.1:44262
I20260923 20:51:25.295344 23418 tablet_service.cc:1514] Processing DeleteTablet for tablet e499dd7e01a545c5a9638269090dc97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:25 UTC) from {username='slave'} at 127.0.0.1:48134
I20260923 20:51:25.295344 23287 tablet_service.cc:1514] Processing DeleteTablet for tablet e499dd7e01a545c5a9638269090dc97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:25 UTC) from {username='slave'} at 127.0.0.1:33022
I20260923 20:51:25.297747 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23225
I20260923 20:51:25.306191 23656 tablet_replica.cc:330] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: stopping tablet replica
I20260923 20:51:25.306932 23656 raft_consensus.cc:2229] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:25.307717 23655 tablet_replica.cc:330] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: stopping tablet replica
I20260923 20:51:25.308300 23656 raft_consensus.cc:2258] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:25.308804 23655 raft_consensus.cc:2229] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:25.309612 23655 raft_consensus.cc:2258] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:25.311678 23656 ts_tablet_manager.cc:1905] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:25.313026 23657 tablet_replica.cc:330] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: stopping tablet replica
I20260923 20:51:25.313966 23655 ts_tablet_manager.cc:1905] T e499dd7e01a545c5a9638269090dc97e P 80fa979d84e84f21a8fac72495334c95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:25.315760 23657 raft_consensus.cc:2229] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:51:25.316597 23657 raft_consensus.cc:2258] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:25.314848 23630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.298850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.5.71.129:44299)
0923 20:51:25.299049 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.299056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.304747 (+ 5691us) server_negotiation.cc:407] Connection header received
0923 20:51:25.305851 (+ 1104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.305858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.305981 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.306065 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:25.307226 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.308473 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.310488 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.311108 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.311219 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.312866 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:25.312879 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:25.312898 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:25.313738 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:25.313803 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:25.313833 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:25.313965 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.314199 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.314326 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:51:25.317875 23347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:25.318812 23657 ts_tablet_manager.cc:1905] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:25.323621 23656 ts_tablet_manager.cc:1918] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:25.324007 23656 log.cc:1196] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/e499dd7e01a545c5a9638269090dc97e
I20260923 20:51:25.324644 23656 ts_tablet_manager.cc:1939] T e499dd7e01a545c5a9638269090dc97e P 802fb6b8314a44bc80f9610f20bfd350: Deleting consensus metadata
I20260923 20:51:25.326483 23657 ts_tablet_manager.cc:1918] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:25.326665 23657 log.cc:1196] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/e499dd7e01a545c5a9638269090dc97e
I20260923 20:51:25.327023 23149 catalog_manager.cc:4841] TS 802fb6b8314a44bc80f9610f20bfd350 (127.5.71.130:46291): tablet e499dd7e01a545c5a9638269090dc97e (table test-table [id=9dbdd4b546614522bb5501da16f2a703]) successfully deleted
I20260923 20:51:25.327310 23657 ts_tablet_manager.cc:1939] T e499dd7e01a545c5a9638269090dc97e P 6ad9512e562e46feb5f103ef42875f2d: Deleting consensus metadata
I20260923 20:51:25.328977 23152 catalog_manager.cc:4841] TS 6ad9512e562e46feb5f103ef42875f2d (127.5.71.131:44369): tablet e499dd7e01a545c5a9638269090dc97e (table test-table [id=9dbdd4b546614522bb5501da16f2a703]) successfully deleted
I20260923 20:51:25.420310 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23225
I20260923 20:51:25.438954 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23356
W20260923 20:51:25.439087 23152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:44299 (error 108)
I20260923 20:51:25.441047 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.440615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33032)
0923 20:51:25.440798 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.440904 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260923 20:51:25.441403 23152 catalog_manager.cc:4584] TS 80fa979d84e84f21a8fac72495334c95 (127.5.71.129:44299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e499dd7e01a545c5a9638269090dc97e: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
I20260923 20:51:25.448410 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.440642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.5.71.130:46291)
0923 20:51:25.440836 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.440843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.440864 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:51:25.440938 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.440947 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.441061 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.441172 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:25.441847 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.442601 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.444833 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.445270 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.445382 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.446743 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:25.446754 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:25.446771 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:25.447416 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:25.447470 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:25.447493 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:25.447623 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.447897 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.448010 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260923 20:51:25.449221 23478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:25.495270 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.494627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33038)
0923 20:51:25.494910 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.495049 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260923 20:51:25.496613 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.496167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33044)
0923 20:51:25.496345 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.496457 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 20:51:25.541256 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23356
I20260923 20:51:25.557752 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.557100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33046)
0923 20:51:25.557367 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.557497 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260923 20:51:25.559062 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.558631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33052)
0923 20:51:25.558812 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.558912 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 20:51:25.560672 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23486
I20260923 20:51:25.569453 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.561591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.5.71.131:44369)
0923 20:51:25.561825 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:25.561829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:25.561850 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:51:25.561915 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:25.561918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:25.561996 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:25.562096 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:25.562606 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.563489 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.565637 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:25.566158 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:25.566268 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:25.567681 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:25.567692 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:25.567712 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:25.568508 (+ 796us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:25.568571 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:25.568611 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:25.568752 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:25.568927 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 20:51:25.569027 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 20:51:25.570339 23608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:25.639823 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.639200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33056)
0923 20:51:25.639444 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.639557 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260923 20:51:25.641132 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.640603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33066)
0923 20:51:25.640847 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.640939 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260923 20:51:25.698838 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23486
I20260923 20:51:25.718168 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23134
I20260923 20:51:25.719350 23195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:25.833408 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:25.832478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:44299 (local address 127.0.0.1:33076)
0923 20:51:25.832994 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:25.833133 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:44299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343}
I20260923 20:51:25.833596 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23134
2026-09-23T20:51:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2995 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:25 dist-test-slave-dn29 krb5kdc[23666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:25 dist-test-slave-dn29 krb5kdc[23666](info): set up 2 sockets
Sep 23 20:51:25 dist-test-slave-dn29 krb5kdc[23666](info): commencing operation
krb5kdc: starting...
W20260923 20:51:27.032830 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:27 dist-test-slave-dn29 krb5kdc[23666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:51:27.304083 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38161
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:27.649128 23679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:27.649435 23679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:27.649613 23679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:27.657099 23679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:27.657232 23679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:27.657326 23679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:27.657413 23679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:27.657500 23679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:27.667659 23679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:27.669219 23679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:27.670593 23679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:27.683440 23684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:27.683552 23687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:27.683524 23685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:27.684458 23679 server_base.cc:1034] running on GCE node
I20260923 20:51:27.685338 23679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:27.686975 23679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:27.688232 23679 hybrid_clock.cc:648] HybridClock initialized: now 1790196687688174 us; error 53 us; skew 500 ppm
I20260923 20:51:27.688597 23679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:27 dist-test-slave-dn29 krb5kdc[23666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196687, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:27.691915 23679 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:51:27.693569 23679 webserver.cc:460] Webserver started at http://127.5.71.190:46399/ using document root <none> and password file <none>
I20260923 20:51:27.694293 23679 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:27.694440 23679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:27.694759 23679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:27.697986 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "86d07b3379b64d2da870b3e4e78d21cb"
format_stamp: "Formatted at 2026-09-23 20:51:27 on dist-test-slave-dn29"
server_key: "49ccb589d8a654c84574734461a89a16"
server_key_iv: "5a9dc10c8e9bbe0b01d67d106a031160"
server_key_version: "encryptionkey@0"
I20260923 20:51:27.698823 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "86d07b3379b64d2da870b3e4e78d21cb"
format_stamp: "Formatted at 2026-09-23 20:51:27 on dist-test-slave-dn29"
server_key: "49ccb589d8a654c84574734461a89a16"
server_key_iv: "5a9dc10c8e9bbe0b01d67d106a031160"
server_key_version: "encryptionkey@0"
I20260923 20:51:27.704682 23679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:51:27.708666 23694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:27.710273 23679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:27.710500 23679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "86d07b3379b64d2da870b3e4e78d21cb"
format_stamp: "Formatted at 2026-09-23 20:51:27 on dist-test-slave-dn29"
server_key: "49ccb589d8a654c84574734461a89a16"
server_key_iv: "5a9dc10c8e9bbe0b01d67d106a031160"
server_key_version: "encryptionkey@0"
I20260923 20:51:27.710817 23679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:27.756693 23679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:27.761611 23679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:27.761970 23679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:27.781610 23679 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:38161
I20260923 20:51:27.781632 23745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:38161 every 8 connection(s)
I20260923 20:51:27.783143 23679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:27.787070 23746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:27.791510 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23679
I20260923 20:51:27.791743 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:27.792318 5406 external_mini_cluster.cc:1428] Setting key 63e69fa3f28c7ee26f5e596e4b82b03c
I20260923 20:51:27.799147 23746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb: Bootstrap starting.
I20260923 20:51:27.802954 23746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:27.804271 23746 log.cc:824] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:27.807420 23746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb: No bootstrap required, opened a new log
Sep 23 20:51:27 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:27.819211 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:27.794335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.5.71.190:38161)
0923 20:51:27.795137 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:27.795150 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:27.795194 (+ 44us) server_negotiation.cc:407] Connection header received
0923 20:51:27.797650 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:27.797676 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:27.798091 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:27.798429 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:27.800338 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:27.801660 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:27.803567 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:27.804581 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:27.804768 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:27.809058 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:27.809071 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:27.809093 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:27.809156 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:27.813883 (+ 4727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:27.814723 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:27.814733 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:27.814744 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:27.814993 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:27.815445 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:27.815452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:27.815458 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:27.816301 (+ 843us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:27.816680 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:27.817001 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 20:51:27.817548 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":394,"threads_started":1}
I20260923 20:51:27.820292 23746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d07b3379b64d2da870b3e4e78d21cb" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38161 } }
I20260923 20:51:27.820801 23746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:27.820966 23746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d07b3379b64d2da870b3e4e78d21cb, State: Initialized, Role: FOLLOWER
I20260923 20:51:27.821588 23746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d07b3379b64d2da870b3e4e78d21cb" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38161 } }
I20260923 20:51:27.821897 23746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:27.822065 23746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:27.822300 23746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:27.824817 23746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d07b3379b64d2da870b3e4e78d21cb" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38161 } }
I20260923 20:51:27.825367 23746 leader_election.cc:302] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86d07b3379b64d2da870b3e4e78d21cb; no voters:
I20260923 20:51:27.826067 23746 leader_election.cc:288] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:27.826169 23751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:27.827085 23751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [term 1 LEADER]: Becoming Leader. State: Replica: 86d07b3379b64d2da870b3e4e78d21cb, State: Running, Role: LEADER
I20260923 20:51:27.827731 23751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d07b3379b64d2da870b3e4e78d21cb" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38161 } }
I20260923 20:51:27.828691 23746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:27.831022 23752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86d07b3379b64d2da870b3e4e78d21cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d07b3379b64d2da870b3e4e78d21cb" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38161 } } }
I20260923 20:51:27.831364 23752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:27.832454 23757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:27.832255 23753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86d07b3379b64d2da870b3e4e78d21cb. Latest consensus state: current_term: 1 leader_uuid: "86d07b3379b64d2da870b3e4e78d21cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86d07b3379b64d2da870b3e4e78d21cb" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38161 } } }
I20260923 20:51:27.832636 23753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:27.837972 23757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:27.849579 23757 catalog_manager.cc:1347] Generated new cluster ID: 5ddd7df39a384c3ab1869ccd27f4b62a
I20260923 20:51:27.849723 23757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:27.862038 23757 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:27.863584 23757 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:27.876897 23757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb: Generated new TSK 0
I20260923 20:51:27.878108 23757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:51:27.948771 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38161
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:28.322662 23772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:28.323027 23772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:28.323220 23772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:28.333002 23772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:28.333190 23772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:28.333393 23772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:28.344969 23772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:28.346345 23772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:28.347779 23772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:28.363409 23777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:28.363305 23778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:28.364385 23780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:28.364480 23772 server_base.cc:1034] running on GCE node
I20260923 20:51:28.365326 23772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:28.366549 23772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:28.367902 23772 hybrid_clock.cc:648] HybridClock initialized: now 1790196688367797 us; error 83 us; skew 500 ppm
I20260923 20:51:28.368283 23772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:28 dist-test-slave-dn29 krb5kdc[23666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:28.372413 23772 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:51:28.374315 23772 webserver.cc:460] Webserver started at http://127.5.71.129:34361/ using document root <none> and password file <none>
I20260923 20:51:28.375080 23772 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:28.375242 23772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:28.375593 23772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:28.378834 23772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "e193f6b6f4a94263905a8a461f7ec9be"
format_stamp: "Formatted at 2026-09-23 20:51:28 on dist-test-slave-dn29"
server_key: "08b4f8fc99ac26b56ba8fde95747b68b"
server_key_iv: "bbfd2cf75d98684e191d8d1a4ac460a0"
server_key_version: "encryptionkey@0"
I20260923 20:51:28.379737 23772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "e193f6b6f4a94263905a8a461f7ec9be"
format_stamp: "Formatted at 2026-09-23 20:51:28 on dist-test-slave-dn29"
server_key: "08b4f8fc99ac26b56ba8fde95747b68b"
server_key_iv: "bbfd2cf75d98684e191d8d1a4ac460a0"
server_key_version: "encryptionkey@0"
I20260923 20:51:28.385483 23772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260923 20:51:28.389495 23787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:28.391088 23772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:28.391321 23772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "e193f6b6f4a94263905a8a461f7ec9be"
format_stamp: "Formatted at 2026-09-23 20:51:28 on dist-test-slave-dn29"
server_key: "08b4f8fc99ac26b56ba8fde95747b68b"
server_key_iv: "bbfd2cf75d98684e191d8d1a4ac460a0"
server_key_version: "encryptionkey@0"
I20260923 20:51:28.391543 23772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:28.417475 23772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:28.426604 23772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:28.426963 23772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:28.428297 23772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:28.429888 23772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:28.429966 23772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:28.430051 23772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:28.430094 23772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:28.467587 23772 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:42061
I20260923 20:51:28.467774 23899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:42061 every 8 connection(s)
I20260923 20:51:28.469080 23772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:28.476831 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23772
I20260923 20:51:28.477044 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:51:28.477559 5406 external_mini_cluster.cc:1428] Setting key 229ed2d6b3860c9f4182d7c37d6d9ca1
Sep 23 20:51:28 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:28.494244 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:28.471972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:51363 (local address 127.5.71.190:38161)
0923 20:51:28.472555 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:28.472561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:28.473677 (+ 1116us) server_negotiation.cc:407] Connection header received
0923 20:51:28.475931 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:28.475936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:28.476010 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:28.476112 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:28.478126 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:28.479042 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:28.480991 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:28.481793 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:28.481951 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:28.486091 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:28.486101 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:28.486116 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:28.486169 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:28.489864 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:28.491139 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:28.491144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:28.491146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:28.491319 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:28.492059 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:28.492064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:28.492066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:28.492373 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:28.492503 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:28.493589 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0923 20:51:28.493963 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":306,"threads_started":1}
I20260923 20:51:28.495729 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:28.472554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38161 (local address 127.5.71.129:51363)
0923 20:51:28.473487 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:28.473530 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:28.475419 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:28.476386 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:28.476395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:28.477142 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:28.477906 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:28.477931 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:28.479217 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:28.479222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:28.480820 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:28.480835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:28.482041 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:28.482047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:28.482280 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:28.483087 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:28.483114 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:28.485760 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:28.490065 (+ 4305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:28.490079 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:28.490103 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:28.490866 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:28.491461 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:28.491469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:28.491476 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:28.491887 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:28.492524 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:28.492537 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:28.493093 (+ 556us) client_negotiation.cc:769] Sending connection context
0923 20:51:28.493424 (+ 331us) client_negotiation.cc:240] Negotiation successful
0923 20:51:28.494138 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":326,"threads_started":1}
I20260923 20:51:28.497649 23900 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38161
I20260923 20:51:28.497999 23900 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:28.498767 23900 heartbeater.cc:511] Master 127.5.71.190:38161 requested a full tablet report, sending...
I20260923 20:51:28.500967 23711 ts_manager.cc:194] Registered new tserver with Master: e193f6b6f4a94263905a8a461f7ec9be (127.5.71.129:42061)
I20260923 20:51:28.502565 23711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:51363
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:51:28.544902 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38161
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:28.884948 23906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:28.885326 23906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:28.885500 23906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:28.893280 23906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:28.893452 23906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:28.893630 23906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:28.904268 23906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:28.905616 23906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:28.906952 23906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:28.921356 23912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:28.921443 23911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:28.921327 23914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:28.922996 23906 server_base.cc:1034] running on GCE node
I20260923 20:51:28.923712 23906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:28.924780 23906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:28.926064 23906 hybrid_clock.cc:648] HybridClock initialized: now 1790196688926000 us; error 50 us; skew 500 ppm
I20260923 20:51:28.926470 23906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:28 dist-test-slave-dn29 krb5kdc[23666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:28.930380 23906 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:51:28.932117 23906 webserver.cc:460] Webserver started at http://127.5.71.130:35361/ using document root <none> and password file <none>
I20260923 20:51:28.932860 23906 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:28.933013 23906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:28.933355 23906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:28.936417 23906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "5de18669236645e397ce4072e8b05539"
format_stamp: "Formatted at 2026-09-23 20:51:28 on dist-test-slave-dn29"
server_key: "f5ca1557e1a22e1c8c3af73839acc1fd"
server_key_iv: "f3e113a54aa54a8d9a8d1f841991d845"
server_key_version: "encryptionkey@0"
I20260923 20:51:28.937271 23906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "5de18669236645e397ce4072e8b05539"
format_stamp: "Formatted at 2026-09-23 20:51:28 on dist-test-slave-dn29"
server_key: "f5ca1557e1a22e1c8c3af73839acc1fd"
server_key_iv: "f3e113a54aa54a8d9a8d1f841991d845"
server_key_version: "encryptionkey@0"
I20260923 20:51:28.943051 23906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260923 20:51:28.946830 23921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:28.948514 23906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:28.948745 23906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "5de18669236645e397ce4072e8b05539"
format_stamp: "Formatted at 2026-09-23 20:51:28 on dist-test-slave-dn29"
server_key: "f5ca1557e1a22e1c8c3af73839acc1fd"
server_key_iv: "f3e113a54aa54a8d9a8d1f841991d845"
server_key_version: "encryptionkey@0"
I20260923 20:51:28.949009 23906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:28.968407 23906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:28.973975 23906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:28.974303 23906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:28.975378 23906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:28.976861 23906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:28.976966 23906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:28.977124 23906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:28.977206 23906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:29.013042 23906 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:46057
I20260923 20:51:29.013134 24033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:46057 every 8 connection(s)
I20260923 20:51:29.014343 23906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:29.022466 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 23906
I20260923 20:51:29.022665 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:51:29.023141 5406 external_mini_cluster.cc:1428] Setting key dfe03f7dcb880436a610dd121386ebd7
Sep 23 20:51:29 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:29.036640 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.017217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:45165 (local address 127.5.71.190:38161)
0923 20:51:29.017673 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.017678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.018654 (+ 976us) server_negotiation.cc:407] Connection header received
0923 20:51:29.020564 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.020569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.020640 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.020740 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:29.022436 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.023385 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.025472 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.025911 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.026011 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.030526 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:29.030531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:29.030534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:29.030567 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:29.033321 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.034081 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.034087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.034092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.034230 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.034748 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.034752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.034754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.035057 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:29.035194 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.036183 (+ 989us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.036365 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":220,"threads_started":1}
I20260923 20:51:29.038192 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.017794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38161 (local address 127.5.71.130:45165)
0923 20:51:29.018491 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.018532 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.020292 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.021005 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.021013 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.021545 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:29.022242 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.022270 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.023550 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.023556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.025311 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.025326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.026167 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.026177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.026503 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.027261 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:29.027304 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:29.030290 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:29.033475 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:29.033485 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:29.033504 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:29.033946 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:29.034343 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:29.034349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:29.034352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:29.034605 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:29.035189 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:29.035200 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:29.035687 (+ 487us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.036042 (+ 355us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.036649 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":287,"threads_started":1}
I20260923 20:51:29.040004 24034 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38161
I20260923 20:51:29.040342 24034 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:29.041101 24034 heartbeater.cc:511] Master 127.5.71.190:38161 requested a full tablet report, sending...
I20260923 20:51:29.042778 23711 ts_manager.cc:194] Registered new tserver with Master: 5de18669236645e397ce4072e8b05539 (127.5.71.130:46057)
I20260923 20:51:29.044180 23711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:45165
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:51:29.084867 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38161
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:29.423449 24040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:29.423755 24040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:29.423943 24040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:29.431463 24040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:29.431584 24040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:29.431701 24040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:29.442018 24040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:29.443472 24040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:29.444883 24040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:29.458659 24048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:29.458709 24046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:29.458709 24045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:29.459606 24040 server_base.cc:1034] running on GCE node
I20260923 20:51:29.460402 24040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:29.461472 24040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:29.462755 24040 hybrid_clock.cc:648] HybridClock initialized: now 1790196689462665 us; error 77 us; skew 500 ppm
I20260923 20:51:29.463122 24040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:29 dist-test-slave-dn29 krb5kdc[23666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196689, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:29.466691 24040 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:51:29.468763 24040 webserver.cc:460] Webserver started at http://127.5.71.131:44633/ using document root <none> and password file <none>
I20260923 20:51:29.469671 24040 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:29.469821 24040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:29.470201 24040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:29.474432 24040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "cf54ee404f9944d6ae81bf992d5873cb"
format_stamp: "Formatted at 2026-09-23 20:51:29 on dist-test-slave-dn29"
server_key: "2670ae804a6fe79017503184d3835189"
server_key_iv: "e766d003946faa45b2dbb3534d8fab23"
server_key_version: "encryptionkey@0"
I20260923 20:51:29.475255 24040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "cf54ee404f9944d6ae81bf992d5873cb"
format_stamp: "Formatted at 2026-09-23 20:51:29 on dist-test-slave-dn29"
server_key: "2670ae804a6fe79017503184d3835189"
server_key_iv: "e766d003946faa45b2dbb3534d8fab23"
server_key_version: "encryptionkey@0"
I20260923 20:51:29.480997 24040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260923 20:51:29.485368 24055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:29.487195 24040 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:51:29.487426 24040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "cf54ee404f9944d6ae81bf992d5873cb"
format_stamp: "Formatted at 2026-09-23 20:51:29 on dist-test-slave-dn29"
server_key: "2670ae804a6fe79017503184d3835189"
server_key_iv: "e766d003946faa45b2dbb3534d8fab23"
server_key_version: "encryptionkey@0"
I20260923 20:51:29.487687 24040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:29.505908 23900 heartbeater.cc:503] Master 127.5.71.190:38161 was elected leader, sending a full tablet report...
I20260923 20:51:29.515981 24040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:29.521224 24040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:29.521577 24040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:29.522743 24040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:29.524526 24040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:29.524646 24040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:29.524817 24040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:29.524899 24040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:29.558066 24040 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:43945
I20260923 20:51:29.558118 24167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:43945 every 8 connection(s)
I20260923 20:51:29.559489 24040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:29.562647 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24040
I20260923 20:51:29.562886 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:51:29.563349 5406 external_mini_cluster.cc:1428] Setting key 0c5a84aa6045cdba3d7a1baef9a97ba3
Sep 23 20:51:29 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196689, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:29.581578 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.562468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:50903 (local address 127.5.71.190:38161)
0923 20:51:29.563025 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.563031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.564041 (+ 1010us) server_negotiation.cc:407] Connection header received
0923 20:51:29.566147 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.566152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.566275 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.566394 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:29.568069 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.568871 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.570435 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.570834 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.570931 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.574909 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:29.574914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:29.574917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:29.574950 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:29.578258 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.579314 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.579320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.579324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.579468 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.580004 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.580008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.580010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.580299 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:29.580428 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.581140 (+ 712us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.581306 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":282,"threads_started":1}
I20260923 20:51:29.582618 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.562973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38161 (local address 127.5.71.131:50903)
0923 20:51:29.563852 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.563906 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.565872 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.566678 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.566686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.567225 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:29.567881 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.567902 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.569036 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.569039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.570298 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.570310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.571104 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.571110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.571345 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.572224 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:29.572253 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:29.574513 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:29.578462 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:29.578474 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:29.578500 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:29.579160 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:29.579586 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:29.579591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:29.579594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:29.579858 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:29.580439 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:29.580445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:29.580813 (+ 368us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.581056 (+ 243us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.581461 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":359,"threads_started":1}
I20260923 20:51:29.584151 24168 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38161
I20260923 20:51:29.584515 24168 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:29.585373 24168 heartbeater.cc:511] Master 127.5.71.190:38161 requested a full tablet report, sending...
I20260923 20:51:29.587085 23711 ts_manager.cc:194] Registered new tserver with Master: cf54ee404f9944d6ae81bf992d5873cb (127.5.71.131:43945)
I20260923 20:51:29.587987 23711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:50903
I20260923 20:51:29.592073 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:51:29.609416 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.596445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55848 (local address 127.5.71.190:38161)
0923 20:51:29.596677 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.596682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.597028 (+ 346us) server_negotiation.cc:407] Connection header received
0923 20:51:29.597175 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.597179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.597246 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.597343 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:29.598889 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.599631 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.601008 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.601427 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.601526 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.604117 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:29.604122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:29.604125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:29.604157 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:29.606541 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.607348 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.607352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.607354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.607477 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.608061 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.608065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.608067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.608372 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:29.608567 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.609005 (+ 438us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.609154 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260923 20:51:29.613634 23711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:51:29.621062 23711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:51:29.645998 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.636165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:43945 (local address 127.0.0.1:57130)
0923 20:51:29.637059 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.637076 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.637215 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.638238 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.638242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.638274 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:29.638615 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.638624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.640200 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.640208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.642840 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.642858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.644761 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.644768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.644975 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.645041 (+ 66us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.645208 (+ 167us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.645465 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":163,"spinlock_wait_cycles":6912,"thread_start_us":687,"threads_started":1}
I20260923 20:51:29.646232 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.634460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.5.71.130:46057)
0923 20:51:29.636580 (+ 2120us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.636587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.636614 (+ 27us) server_negotiation.cc:407] Connection header received
0923 20:51:29.636720 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.636730 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.636937 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.638042 (+ 1105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:29.638273 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.639748 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.643462 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.645029 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.645168 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.645349 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.645700 (+ 351us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.645895 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1975,"thread_start_us":1863,"threads_started":1}
I20260923 20:51:29.646802 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.635379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39878 (local address 127.5.71.129:42061)
0923 20:51:29.636184 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.636195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.636225 (+ 30us) server_negotiation.cc:407] Connection header received
0923 20:51:29.636336 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.636346 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.636608 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.636829 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:29.637939 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.639874 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.643531 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.645028 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.645145 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.645382 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.646404 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.646567 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":1138,"threads_started":1}
I20260923 20:51:29.646919 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.636677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57130 (local address 127.5.71.131:43945)
0923 20:51:29.637714 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.637723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.637766 (+ 43us) server_negotiation.cc:407] Connection header received
0923 20:51:29.637860 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.637869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.638067 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.638249 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:29.638810 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.640070 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.642997 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.644555 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.644696 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.644960 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.646497 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.646654 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":685,"threads_started":1}
I20260923 20:51:29.648038 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.634384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:46057 (local address 127.0.0.1:47234)
0923 20:51:29.635304 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.635341 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.635517 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.637130 (+ 1613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.637133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.637185 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:29.637819 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.637832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.639943 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.639951 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.643243 (+ 3292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.643259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.645266 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.645272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.645482 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.645514 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.645613 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.645746 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":152,"spinlock_wait_cycles":24192,"thread_start_us":545,"threads_started":1}
I20260923 20:51:29.650507 24103 tablet_service.cc:1467] Processing CreateTablet for tablet 4760f9624f37469aa5add7d7397b3144 (DEFAULT_TABLE table=test-table [id=930fad922b67415d8f3e75abbd2dc87b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:29.651250 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.634634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39878)
0923 20:51:29.635987 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.636006 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.636113 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.637106 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.637115 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.637146 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:29.637749 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.637761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.640031 (+ 2270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.640035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.643355 (+ 3320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.643369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.645560 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.645564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.645763 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.645798 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.645895 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.646043 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":517,"thread_start_us":1082,"threads_started":1}
I20260923 20:51:29.651165 23969 tablet_service.cc:1467] Processing CreateTablet for tablet 4760f9624f37469aa5add7d7397b3144 (DEFAULT_TABLE table=test-table [id=930fad922b67415d8f3e75abbd2dc87b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:29.652071 24103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4760f9624f37469aa5add7d7397b3144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:29.653220 23969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4760f9624f37469aa5add7d7397b3144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:29.655268 23835 tablet_service.cc:1467] Processing CreateTablet for tablet 4760f9624f37469aa5add7d7397b3144 (DEFAULT_TABLE table=test-table [id=930fad922b67415d8f3e75abbd2dc87b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:29.656837 23835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4760f9624f37469aa5add7d7397b3144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:29.664987 24187 tablet_bootstrap.cc:492] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Bootstrap starting.
I20260923 20:51:29.668982 24187 tablet_bootstrap.cc:654] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:29.670045 24187 log.cc:824] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:29.673342 24187 tablet_bootstrap.cc:492] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: No bootstrap required, opened a new log
I20260923 20:51:29.673496 24190 tablet_bootstrap.cc:492] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Bootstrap starting.
I20260923 20:51:29.673667 24187 ts_tablet_manager.cc:1397] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260923 20:51:29.674867 24188 tablet_bootstrap.cc:492] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Bootstrap starting.
I20260923 20:51:29.677399 24190 tablet_bootstrap.cc:654] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:29.678738 24188 tablet_bootstrap.cc:654] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:29.678761 24190 log.cc:824] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:29.680024 24188 log.cc:824] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:29.682147 24190 tablet_bootstrap.cc:492] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: No bootstrap required, opened a new log
I20260923 20:51:29.682466 24190 ts_tablet_manager.cc:1397] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260923 20:51:29.682715 24187 raft_consensus.cc:348] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.683205 24188 tablet_bootstrap.cc:492] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: No bootstrap required, opened a new log
I20260923 20:51:29.683187 24187 raft_consensus.cc:374] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:29.683290 24187 raft_consensus.cc:729] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de18669236645e397ce4072e8b05539, State: Initialized, Role: FOLLOWER
I20260923 20:51:29.683835 24188 ts_tablet_manager.cc:1397] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260923 20:51:29.684113 24187 consensus_queue.cc:260] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.685365 24034 heartbeater.cc:503] Master 127.5.71.190:38161 was elected leader, sending a full tablet report...
I20260923 20:51:29.686062 24187 ts_tablet_manager.cc:1428] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260923 20:51:29.693589 24190 raft_consensus.cc:348] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.694125 24190 raft_consensus.cc:374] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:29.694252 24190 raft_consensus.cc:729] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e193f6b6f4a94263905a8a461f7ec9be, State: Initialized, Role: FOLLOWER
I20260923 20:51:29.694949 24190 consensus_queue.cc:260] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.695405 24188 raft_consensus.cc:348] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.696017 24188 raft_consensus.cc:374] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:29.696116 24188 raft_consensus.cc:729] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf54ee404f9944d6ae81bf992d5873cb, State: Initialized, Role: FOLLOWER
I20260923 20:51:29.696718 24188 consensus_queue.cc:260] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.697383 24190 ts_tablet_manager.cc:1428] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260923 20:51:29.697917 24168 heartbeater.cc:503] Master 127.5.71.190:38161 was elected leader, sending a full tablet report...
I20260923 20:51:29.698442 24188 ts_tablet_manager.cc:1428] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260923 20:51:29.721725 23901 tablet.cc:2369] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:29.730464 24195 raft_consensus.cc:482] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:29.730782 24195 raft_consensus.cc:504] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.732715 24195 leader_election.cc:288] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5de18669236645e397ce4072e8b05539 (127.5.71.130:46057), e193f6b6f4a94263905a8a461f7ec9be (127.5.71.129:42061)
I20260923 20:51:29.739599 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.733183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:54699 (local address 127.5.71.129:42061)
0923 20:51:29.733414 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.733421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.733755 (+ 334us) server_negotiation.cc:407] Connection header received
0923 20:51:29.734016 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.734022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.734116 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.734215 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:29.734785 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.735765 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.737620 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.738753 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.738869 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.738922 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.739299 (+ 377us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.739403 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:51:29.739779 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.733198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.5.71.131:54699)
0923 20:51:29.733603 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.733644 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.733821 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.734247 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.734253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.734301 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:29.734497 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.734512 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.735954 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.735959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.737451 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.737473 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.738906 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.738912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.739105 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.739141 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.739241 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.739379 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
I20260923 20:51:29.740690 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.733314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:33637 (local address 127.5.71.130:46057)
0923 20:51:29.733530 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.733536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.734219 (+ 683us) server_negotiation.cc:407] Connection header received
0923 20:51:29.734372 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.734377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.734454 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.734553 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:29.735429 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.736430 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.738550 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.739651 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.739854 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.739930 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.740359 (+ 429us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.740503 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:51:29.740823 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.733242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:46057 (local address 127.5.71.131:33637)
0923 20:51:29.734077 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.734095 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:29.734233 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:29.734787 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:29.734791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:29.734814 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:29.735248 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.735261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.736651 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.736655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.738380 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:29.738394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.740052 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.740058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:29.740200 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.740222 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 20:51:29.740313 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 20:51:29.740399 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":419,"threads_started":1}
I20260923 20:51:29.741051 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4760f9624f37469aa5add7d7397b3144" candidate_uuid: "cf54ee404f9944d6ae81bf992d5873cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e193f6b6f4a94263905a8a461f7ec9be" is_pre_election: true
I20260923 20:51:29.741719 23855 raft_consensus.cc:2454] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf54ee404f9944d6ae81bf992d5873cb in term 0.
I20260923 20:51:29.742031 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4760f9624f37469aa5add7d7397b3144" candidate_uuid: "cf54ee404f9944d6ae81bf992d5873cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de18669236645e397ce4072e8b05539" is_pre_election: true
I20260923 20:51:29.742609 24058 leader_election.cc:302] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf54ee404f9944d6ae81bf992d5873cb, e193f6b6f4a94263905a8a461f7ec9be; no voters:
I20260923 20:51:29.742622 23989 raft_consensus.cc:2454] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf54ee404f9944d6ae81bf992d5873cb in term 0.
I20260923 20:51:29.743012 24195 raft_consensus.cc:2781] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:51:29.743214 24195 raft_consensus.cc:482] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:51:29.743363 24195 raft_consensus.cc:3037] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:29.746006 24195 raft_consensus.cc:504] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.746654 24195 leader_election.cc:288] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [CANDIDATE]: Term 1 election: Requested vote from peers 5de18669236645e397ce4072e8b05539 (127.5.71.130:46057), e193f6b6f4a94263905a8a461f7ec9be (127.5.71.129:42061)
I20260923 20:51:29.747279 23855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4760f9624f37469aa5add7d7397b3144" candidate_uuid: "cf54ee404f9944d6ae81bf992d5873cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e193f6b6f4a94263905a8a461f7ec9be"
I20260923 20:51:29.747294 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4760f9624f37469aa5add7d7397b3144" candidate_uuid: "cf54ee404f9944d6ae81bf992d5873cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de18669236645e397ce4072e8b05539"
I20260923 20:51:29.747504 23855 raft_consensus.cc:3037] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:29.747511 23989 raft_consensus.cc:3037] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:29.749833 23989 raft_consensus.cc:2454] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf54ee404f9944d6ae81bf992d5873cb in term 1.
I20260923 20:51:29.749845 23855 raft_consensus.cc:2454] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf54ee404f9944d6ae81bf992d5873cb in term 1.
I20260923 20:51:29.750303 24056 leader_election.cc:302] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de18669236645e397ce4072e8b05539, cf54ee404f9944d6ae81bf992d5873cb; no voters:
I20260923 20:51:29.750645 24195 raft_consensus.cc:2781] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:29.751037 24195 raft_consensus.cc:686] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 1 LEADER]: Becoming Leader. State: Replica: cf54ee404f9944d6ae81bf992d5873cb, State: Running, Role: LEADER
I20260923 20:51:29.751601 24195 consensus_queue.cc:237] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } }
I20260923 20:51:29.756371 23711 catalog_manager.cc:5495] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb reported cstate change: term changed from 0 to 1, leader changed from <none> to cf54ee404f9944d6ae81bf992d5873cb (127.5.71.131). New cstate: current_term: 1 leader_uuid: "cf54ee404f9944d6ae81bf992d5873cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf54ee404f9944d6ae81bf992d5873cb" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 43945 } health_report { overall_health: HEALTHY } } }
W20260923 20:51:29.766417 24035 tablet.cc:2369] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:29.781764 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.775871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.5.71.131:43945)
0923 20:51:29.776046 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.776051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.776069 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:51:29.776246 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.776249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.776340 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.776472 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:51:29.776998 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.777775 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.779276 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.779722 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.779886 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.780491 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:51:29.781044 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:51:29.781204 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.781392 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.781458 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 20:51:29.790162 23989 raft_consensus.cc:1261] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 1 FOLLOWER]: Refusing update from remote peer cf54ee404f9944d6ae81bf992d5873cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:29.790349 23855 raft_consensus.cc:1261] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 1 FOLLOWER]: Refusing update from remote peer cf54ee404f9944d6ae81bf992d5873cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:29.791222 24197 consensus_queue.cc:1035] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e193f6b6f4a94263905a8a461f7ec9be" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 42061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:29.791756 24195 consensus_queue.cc:1035] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de18669236645e397ce4072e8b05539" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:29.806919 24201 mvcc.cc:205] Tried to move back new op lower bound from 7332645641369743360 to 7332645641226518528. Current Snapshot: MvccSnapshot[applied={T|T < 7332645641369743360}]
I20260923 20:51:29.823937 23710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:29.824221 23710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:29.828358 23710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86d07b3379b64d2da870b3e4e78d21cb: Sending DeleteTablet for 3 replicas of tablet 4760f9624f37469aa5add7d7397b3144
I20260923 20:51:29.829734 23969 tablet_service.cc:1514] Processing DeleteTablet for tablet 4760f9624f37469aa5add7d7397b3144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:29 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:47234
I20260923 20:51:29.829836 23835 tablet_service.cc:1514] Processing DeleteTablet for tablet 4760f9624f37469aa5add7d7397b3144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:29 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:39878
I20260923 20:51:29.830778 24103 tablet_service.cc:1514] Processing DeleteTablet for tablet 4760f9624f37469aa5add7d7397b3144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:29 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:57130
I20260923 20:51:29.831355 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23772
I20260923 20:51:29.842015 24210 tablet_replica.cc:330] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: stopping tablet replica
I20260923 20:51:29.842641 24210 raft_consensus.cc:2229] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:29.843289 24210 raft_consensus.cc:2258] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:29.846024 24210 ts_tablet_manager.cc:1905] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:29.855014 24210 ts_tablet_manager.cc:1918] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:29.855212 24210 log.cc:1196] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/4760f9624f37469aa5add7d7397b3144
I20260923 20:51:29.855921 24210 ts_tablet_manager.cc:1939] T 4760f9624f37469aa5add7d7397b3144 P 5de18669236645e397ce4072e8b05539: Deleting consensus metadata
Sep 23 20:51:29 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:51:29.858222 24211 tablet_replica.cc:330] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: stopping tablet replica
I20260923 20:51:29.858167 23695 catalog_manager.cc:4841] TS 5de18669236645e397ce4072e8b05539 (127.5.71.130:46057): tablet 4760f9624f37469aa5add7d7397b3144 (table test-table [id=930fad922b67415d8f3e75abbd2dc87b]) successfully deleted
I20260923 20:51:29.858705 24211 raft_consensus.cc:2229] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:51:29.859423 24211 raft_consensus.cc:2258] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:29.860795 24209 tablet_replica.cc:330] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: stopping tablet replica
I20260923 20:51:29.861284 24209 raft_consensus.cc:2229] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:29.861862 24209 raft_consensus.cc:2258] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:29.862658 24211 ts_tablet_manager.cc:1905] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:29.864217 24209 ts_tablet_manager.cc:1905] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:29.868230 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.832637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39886 (local address 127.5.71.129:42061)
0923 20:51:29.832834 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.832841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.844837 (+ 11996us) server_negotiation.cc:407] Connection header received
0923 20:51:29.845387 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.845394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.845510 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.845595 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:29.847661 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.849067 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.851306 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.852099 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.852241 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:29.859232 (+ 6991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:29.859255 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:29.859274 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:29.859329 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:29.863848 (+ 4519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.864879 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.864888 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.864892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.865068 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:29.865526 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:29.865531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:29.865534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:29.866001 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:29.866316 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:29.867521 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0923 20:51:29.867826 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:51:29.869673 23895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:29.872895 24211 ts_tablet_manager.cc:1918] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:29.873308 24211 log.cc:1196] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/4760f9624f37469aa5add7d7397b3144
I20260923 20:51:29.874408 24211 ts_tablet_manager.cc:1939] T 4760f9624f37469aa5add7d7397b3144 P cf54ee404f9944d6ae81bf992d5873cb: Deleting consensus metadata
I20260923 20:51:29.877043 23697 catalog_manager.cc:4841] TS cf54ee404f9944d6ae81bf992d5873cb (127.5.71.131:43945): tablet 4760f9624f37469aa5add7d7397b3144 (table test-table [id=930fad922b67415d8f3e75abbd2dc87b]) successfully deleted
I20260923 20:51:29.964123 24209 ts_tablet_manager.cc:1918] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:29.964389 24209 log.cc:1196] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/4760f9624f37469aa5add7d7397b3144
I20260923 20:51:29.964962 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23772
I20260923 20:51:29.965020 24209 ts_tablet_manager.cc:1939] T 4760f9624f37469aa5add7d7397b3144 P e193f6b6f4a94263905a8a461f7ec9be: Deleting consensus metadata
W20260923 20:51:29.985049 23697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:42061 (error 108)
I20260923 20:51:29.985880 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23906
I20260923 20:51:29.987558 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.986961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39902)
0923 20:51:29.987267 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:29.987415 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
W20260923 20:51:29.987999 23697 catalog_manager.cc:4584] TS e193f6b6f4a94263905a8a461f7ec9be (127.5.71.129:42061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4760f9624f37469aa5add7d7397b3144: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Sep 23 20:51:29 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:51:30.008554 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:29.987199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.5.71.130:46057)
0923 20:51:29.987420 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:29.987428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:29.987451 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:51:29.987530 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:29.987536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:29.987634 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:29.987748 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:29.990062 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.991037 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.993846 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:29.994347 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:29.994498 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:30.000754 (+ 6256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:30.000774 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:30.000794 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:30.000844 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:30.004508 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:30.005628 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:30.005737 (+ 109us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:30.005740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:30.005913 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:30.006515 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:30.006520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:30.006522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:30.007064 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:30.007457 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:30.007922 (+ 465us) server_negotiation.cc:299] Negotiation successful
0923 20:51:30.008119 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260923 20:51:30.009424 24029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:30.042109 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.041447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39910)
0923 20:51:30.041745 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.041868 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260923 20:51:30.043362 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.042900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39922)
0923 20:51:30.043089 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.043202 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260923 20:51:30.104393 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.103601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39928)
0923 20:51:30.103962 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.104102 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260923 20:51:30.105769 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.105414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39942)
0923 20:51:30.105563 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.105645 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 20:51:30.106626 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23906
I20260923 20:51:30.124339 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24040
Sep 23 20:51:30 dist-test-slave-dn29 krb5kdc[23666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:51:30.153599 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.125129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.5.71.131:43945)
0923 20:51:30.125321 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:30.125329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:30.125627 (+ 298us) server_negotiation.cc:407] Connection header received
0923 20:51:30.125805 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:30.125810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:30.125907 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:30.126000 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:30.127982 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:30.128912 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:30.130869 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:30.131367 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:30.131465 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:30.145568 (+ 14103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:30.145588 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:30.145631 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:30.145697 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:30.150037 (+ 4340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:30.150984 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:30.150995 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:30.150999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:30.151196 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:30.151733 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:30.151738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:30.151741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:30.152272 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:30.152631 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:30.152942 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 20:51:30.153188 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 20:51:30.154487 24163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:30.186465 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.185929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39950)
0923 20:51:30.186175 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.186287 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260923 20:51:30.187875 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.187277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39956)
0923 20:51:30.187526 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.187600 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260923 20:51:30.286398 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24040
I20260923 20:51:30.307354 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 23679
I20260923 20:51:30.308637 23741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:30.426949 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.426284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:42061 (local address 127.0.0.1:39958)
0923 20:51:30.426593 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:30.426735 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:42061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260923 20:51:30.426976 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 23679
2026-09-23T20:51:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4597 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T20:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:30Z Disabled control of system clock
I20260923 20:51:30.509356 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40371
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 20:51:30.860388 24221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:30.860684 24221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:30.860826 24221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:30.868340 24221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:30.868459 24221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:30.868528 24221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:30.868615 24221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:30.868707 24221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:30.879001 24221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:30.880513 24221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:30.882391 24221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:30.895120 24226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:30.895196 24227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:30.895150 24229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:30.896059 24221 server_base.cc:1034] running on GCE node
I20260923 20:51:30.896884 24221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:30.898397 24221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:30.899883 24221 hybrid_clock.cc:648] HybridClock initialized: now 1790196690899745 us; error 53 us; skew 500 ppm
I20260923 20:51:30.900314 24221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:30.903203 24221 webserver.cc:460] Webserver started at http://127.5.71.190:34843/ using document root <none> and password file <none>
I20260923 20:51:30.904132 24221 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:30.904304 24221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:30.904721 24221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:30.908392 24221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "576a90e8b4394284a9bc1611b162c138"
format_stamp: "Formatted at 2026-09-23 20:51:30 on dist-test-slave-dn29"
server_key: "41fab7fddcc49fd813c2d523fabde3a1"
server_key_iv: "bafcbbf1a1524905c08dfaa83f175353"
server_key_version: "encryptionkey@0"
I20260923 20:51:30.909276 24221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "576a90e8b4394284a9bc1611b162c138"
format_stamp: "Formatted at 2026-09-23 20:51:30 on dist-test-slave-dn29"
server_key: "41fab7fddcc49fd813c2d523fabde3a1"
server_key_iv: "bafcbbf1a1524905c08dfaa83f175353"
server_key_version: "encryptionkey@0"
I20260923 20:51:30.914830 24221 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:51:30.918812 24235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:30.920511 24221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:30.920730 24221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "576a90e8b4394284a9bc1611b162c138"
format_stamp: "Formatted at 2026-09-23 20:51:30 on dist-test-slave-dn29"
server_key: "41fab7fddcc49fd813c2d523fabde3a1"
server_key_iv: "bafcbbf1a1524905c08dfaa83f175353"
server_key_version: "encryptionkey@0"
I20260923 20:51:30.920986 24221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:30.964409 24221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:30.965356 24221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:30.965687 24221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:30.985829 24221 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:40371
I20260923 20:51:30.985849 24286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:40371 every 8 connection(s)
I20260923 20:51:30.987335 24221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:30.990522 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24221
I20260923 20:51:30.990733 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:30.991325 5406 external_mini_cluster.cc:1428] Setting key 6bd09dd7f6eeb5f239e8ff09d097c98b
I20260923 20:51:30.993013 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:31.006038 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:30.993438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34752 (local address 127.5.71.190:40371)
0923 20:51:30.994461 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:30.994473 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:30.994516 (+ 43us) server_negotiation.cc:407] Connection header received
0923 20:51:30.996432 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:30.996460 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:30.996982 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:30.997439 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:30.998073 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:30.999531 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:31.001211 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:31.001921 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:31.002147 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:31.003462 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:31.003471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:31.003492 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:31.004197 (+ 705us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:31.004279 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:31.004304 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:31.004454 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:31.004704 (+ 250us) server_negotiation.cc:299] Negotiation successful
0923 20:51:31.005047 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":350,"threads_started":1}
I20260923 20:51:31.006927 24287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138: Bootstrap starting.
I20260923 20:51:31.010740 24287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:31.011888 24287 log.cc:824] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:31.015089 24287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138: No bootstrap required, opened a new log
I20260923 20:51:31.023453 24287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576a90e8b4394284a9bc1611b162c138" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40371 } }
I20260923 20:51:31.023941 24287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:31.024040 24287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 576a90e8b4394284a9bc1611b162c138, State: Initialized, Role: FOLLOWER
I20260923 20:51:31.024739 24287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576a90e8b4394284a9bc1611b162c138" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40371 } }
I20260923 20:51:31.025017 24287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:31.025152 24287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:31.025350 24287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:31.028216 24287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576a90e8b4394284a9bc1611b162c138" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40371 } }
I20260923 20:51:31.028751 24287 leader_election.cc:302] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 576a90e8b4394284a9bc1611b162c138; no voters:
I20260923 20:51:31.029352 24287 leader_election.cc:288] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:31.029515 24292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:31.030359 24292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [term 1 LEADER]: Becoming Leader. State: Replica: 576a90e8b4394284a9bc1611b162c138, State: Running, Role: LEADER
I20260923 20:51:31.030977 24292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576a90e8b4394284a9bc1611b162c138" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40371 } }
I20260923 20:51:31.031747 24287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:31.034101 24293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "576a90e8b4394284a9bc1611b162c138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576a90e8b4394284a9bc1611b162c138" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40371 } } }
I20260923 20:51:31.034608 24293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:31.035257 24294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 576a90e8b4394284a9bc1611b162c138. Latest consensus state: current_term: 1 leader_uuid: "576a90e8b4394284a9bc1611b162c138" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576a90e8b4394284a9bc1611b162c138" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40371 } } }
I20260923 20:51:31.035637 24294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:31.037000 24300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:31.042052 24300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:31.051223 24300 catalog_manager.cc:1347] Generated new cluster ID: e8e900ead30246cbaf5a1dc995c53c95
I20260923 20:51:31.051426 24300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:31.095567 24300 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:31.097031 24300 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:31.104543 24300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138: Generated new TSK 0
I20260923 20:51:31.105469 24300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:31.124735 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40371
--builtin_ntp_servers=127.5.71.148:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 20:51:31.475365 24311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:31.475682 24311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:31.475841 24311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:31.483546 24311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:31.483666 24311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:31.483886 24311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:31.494573 24311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:31.496227 24311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:31.498150 24311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:31.512763 24317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:31.512881 24319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:31.512862 24316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:31.513777 24311 server_base.cc:1034] running on GCE node
I20260923 20:51:31.514511 24311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:31.515625 24311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:31.516921 24311 hybrid_clock.cc:648] HybridClock initialized: now 1790196691516858 us; error 87 us; skew 500 ppm
I20260923 20:51:31.517310 24311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:31.520238 24311 webserver.cc:460] Webserver started at http://127.5.71.129:39763/ using document root <none> and password file <none>
I20260923 20:51:31.521003 24311 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:31.521157 24311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:31.521497 24311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:31.525456 24311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "456240576e1e4ced976f7405698daf66"
format_stamp: "Formatted at 2026-09-23 20:51:31 on dist-test-slave-dn29"
server_key: "020f05f86774edd400d470907c42ba6f"
server_key_iv: "b2d8630461b30747b781aa35afcb8ee2"
server_key_version: "encryptionkey@0"
I20260923 20:51:31.526513 24311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "456240576e1e4ced976f7405698daf66"
format_stamp: "Formatted at 2026-09-23 20:51:31 on dist-test-slave-dn29"
server_key: "020f05f86774edd400d470907c42ba6f"
server_key_iv: "b2d8630461b30747b781aa35afcb8ee2"
server_key_version: "encryptionkey@0"
I20260923 20:51:31.532228 24311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260923 20:51:31.536407 24325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:31.537968 24311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:51:31.538117 24311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "456240576e1e4ced976f7405698daf66"
format_stamp: "Formatted at 2026-09-23 20:51:31 on dist-test-slave-dn29"
server_key: "020f05f86774edd400d470907c42ba6f"
server_key_iv: "b2d8630461b30747b781aa35afcb8ee2"
server_key_version: "encryptionkey@0"
I20260923 20:51:31.538317 24311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:31.588057 24311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:31.588939 24311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:31.589222 24311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:31.590212 24311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:31.591892 24311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:31.591984 24311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:31.592057 24311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:31.592108 24311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:31.629820 24311 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:45639
I20260923 20:51:31.629791 24437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:45639 every 8 connection(s)
I20260923 20:51:31.631168 24311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:31.632473 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24311
I20260923 20:51:31.632766 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:51:31.633332 5406 external_mini_cluster.cc:1428] Setting key 28252fd24d5ec7fe2afe5aba56689045
I20260923 20:51:31.636762 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40371
--builtin_ntp_servers=127.5.71.148:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 20:51:31.646677 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:31.634213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:46909 (local address 127.5.71.190:40371)
0923 20:51:31.634684 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:31.634690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:31.636023 (+ 1333us) server_negotiation.cc:407] Connection header received
0923 20:51:31.638215 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:31.638224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:31.638316 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:31.638436 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:31.639522 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:31.640713 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:31.642852 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:31.643644 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:31.643821 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:31.645664 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:31.645669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:31.645673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:31.645694 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:31.645738 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:31.645751 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:31.645866 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:31.646148 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 20:51:31.646264 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":244,"threads_started":1}
I20260923 20:51:31.647927 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:31.634772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40371 (local address 127.5.71.129:46909)
0923 20:51:31.635869 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:31.635923 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:31.637932 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:31.638637 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:31.638645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:31.638737 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:31.639342 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:31.639361 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:31.640943 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:31.640948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:31.642672 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:31.642691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:31.643953 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:31.643958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:31.644260 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:31.645215 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:31.645220 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:31.645283 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:31.645420 (+ 137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:31.645867 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:31.645883 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:31.645897 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 20:51:31.646128 (+ 231us) client_negotiation.cc:240] Negotiation successful
0923 20:51:31.646601 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":361,"threads_started":1}
I20260923 20:51:31.649729 24438 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40371
I20260923 20:51:31.650105 24438 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:31.650954 24438 heartbeater.cc:511] Master 127.5.71.190:40371 requested a full tablet report, sending...
I20260923 20:51:31.653394 24252 ts_manager.cc:194] Registered new tserver with Master: 456240576e1e4ced976f7405698daf66 (127.5.71.129:45639)
I20260923 20:51:31.655067 24252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.129:46909
W20260923 20:51:31.974589 24442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:31.974908 24442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:31.975056 24442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:31.982607 24442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:31.982733 24442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:31.982908 24442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:31.996999 24442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:31.998668 24442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:32.001053 24442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:32.016724 24448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:32.016678 24447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:32.016757 24450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:32.018016 24442 server_base.cc:1034] running on GCE node
I20260923 20:51:32.018723 24442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:32.019814 24442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:32.021106 24442 hybrid_clock.cc:648] HybridClock initialized: now 1790196692021034 us; error 69 us; skew 500 ppm
I20260923 20:51:32.021476 24442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:32.024003 24442 webserver.cc:460] Webserver started at http://127.5.71.130:36589/ using document root <none> and password file <none>
I20260923 20:51:32.024772 24442 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:32.024900 24442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:32.025324 24442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:32.028337 24442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "211b5bac508f47e1b94477e16e8cbb99"
format_stamp: "Formatted at 2026-09-23 20:51:32 on dist-test-slave-dn29"
server_key: "9b0eff488032b0e16e9cb4d0a8e27f3d"
server_key_iv: "a02b372c9ba5aba8f5f06ea79145a00f"
server_key_version: "encryptionkey@0"
I20260923 20:51:32.029131 24442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "211b5bac508f47e1b94477e16e8cbb99"
format_stamp: "Formatted at 2026-09-23 20:51:32 on dist-test-slave-dn29"
server_key: "9b0eff488032b0e16e9cb4d0a8e27f3d"
server_key_iv: "a02b372c9ba5aba8f5f06ea79145a00f"
server_key_version: "encryptionkey@0"
I20260923 20:51:32.034724 24442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260923 20:51:32.038506 24456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:32.039999 24442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:32.040174 24442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "211b5bac508f47e1b94477e16e8cbb99"
format_stamp: "Formatted at 2026-09-23 20:51:32 on dist-test-slave-dn29"
server_key: "9b0eff488032b0e16e9cb4d0a8e27f3d"
server_key_iv: "a02b372c9ba5aba8f5f06ea79145a00f"
server_key_version: "encryptionkey@0"
I20260923 20:51:32.040410 24442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:32.056123 24442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:32.057008 24442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:32.057325 24442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:32.058353 24442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:32.059938 24442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:32.060062 24442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:32.060206 24442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:32.060294 24442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:32.096846 24442 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:45295
I20260923 20:51:32.096942 24568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:45295 every 8 connection(s)
I20260923 20:51:32.098304 24442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:32.105820 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24442
I20260923 20:51:32.106032 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:51:32.106587 5406 external_mini_cluster.cc:1428] Setting key b124d562aa189acb44b69efa82c85517
I20260923 20:51:32.109607 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40371
--builtin_ntp_servers=127.5.71.148:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 20:51:32.112619 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.101334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:51021 (local address 127.5.71.190:40371)
0923 20:51:32.101550 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.101555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.102799 (+ 1244us) server_negotiation.cc:407] Connection header received
0923 20:51:32.105088 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.105094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.105198 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.105324 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:32.106669 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.107704 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.109437 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.109992 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.110139 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.111664 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:32.111672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:32.111677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:32.111707 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:32.111748 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:32.111762 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:32.111905 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.112158 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.112263 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:51:32.114048 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.101633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40371 (local address 127.5.71.130:51021)
0923 20:51:32.102614 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.102652 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.104739 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.105610 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.105619 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.105716 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:32.106485 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.106505 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.107914 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.107925 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.109275 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.109288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.110248 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.110257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.110507 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.111317 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:32.111320 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:32.111362 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:32.111448 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:32.111915 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:32.111929 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:32.111947 (+ 18us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.112149 (+ 202us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.112610 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":314,"threads_started":1}
I20260923 20:51:32.115703 24569 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40371
I20260923 20:51:32.116074 24569 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:32.116925 24569 heartbeater.cc:511] Master 127.5.71.190:40371 requested a full tablet report, sending...
I20260923 20:51:32.118520 24252 ts_manager.cc:194] Registered new tserver with Master: 211b5bac508f47e1b94477e16e8cbb99 (127.5.71.130:45295)
I20260923 20:51:32.119836 24252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.130:51021
W20260923 20:51:32.454087 24572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:32.454474 24572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:32.454643 24572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:32.462076 24572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:32.462247 24572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:32.462426 24572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:32.472144 24572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:40371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:32.473526 24572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:32.475364 24572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:32.489539 24580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:32.489431 24578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:32.489487 24577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:32.490312 24572 server_base.cc:1034] running on GCE node
I20260923 20:51:32.491015 24572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:32.492126 24572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:32.493410 24572 hybrid_clock.cc:648] HybridClock initialized: now 1790196692493360 us; error 47 us; skew 500 ppm
I20260923 20:51:32.493763 24572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:32.496464 24572 webserver.cc:460] Webserver started at http://127.5.71.131:36513/ using document root <none> and password file <none>
I20260923 20:51:32.497272 24572 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:32.497407 24572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:32.497751 24572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:32.500944 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "c79430ac0b954622b6a38310c3427e2f"
format_stamp: "Formatted at 2026-09-23 20:51:32 on dist-test-slave-dn29"
server_key: "5dfcd81ac005c1dec36d4064ef3e8179"
server_key_iv: "045c9a6dabad0acfc028b4e1706b6290"
server_key_version: "encryptionkey@0"
I20260923 20:51:32.501765 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "c79430ac0b954622b6a38310c3427e2f"
format_stamp: "Formatted at 2026-09-23 20:51:32 on dist-test-slave-dn29"
server_key: "5dfcd81ac005c1dec36d4064ef3e8179"
server_key_iv: "045c9a6dabad0acfc028b4e1706b6290"
server_key_version: "encryptionkey@0"
I20260923 20:51:32.507074 24572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260923 20:51:32.511037 24586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:32.513125 24572 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:32.513329 24572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "c79430ac0b954622b6a38310c3427e2f"
format_stamp: "Formatted at 2026-09-23 20:51:32 on dist-test-slave-dn29"
server_key: "5dfcd81ac005c1dec36d4064ef3e8179"
server_key_iv: "045c9a6dabad0acfc028b4e1706b6290"
server_key_version: "encryptionkey@0"
I20260923 20:51:32.513631 24572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:32.526826 24572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:32.527726 24572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:32.528128 24572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:32.529145 24572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:32.530676 24572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:32.530805 24572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:32.530941 24572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:32.531034 24572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:32.566573 24572 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:34613
I20260923 20:51:32.566620 24698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:34613 every 8 connection(s)
I20260923 20:51:32.567945 24572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:32.568984 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24572
I20260923 20:51:32.569149 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:51:32.569720 5406 external_mini_cluster.cc:1428] Setting key 77d6f230ea2febf4e9476a4ec514ab53
I20260923 20:51:32.581169 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.570750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:40735 (local address 127.5.71.190:40371)
0923 20:51:32.570963 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.570972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.572695 (+ 1723us) server_negotiation.cc:407] Connection header received
0923 20:51:32.574427 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.574431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.574499 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.574581 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:32.575592 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.576485 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.578091 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.578534 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.578657 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.580210 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:32.580217 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:32.580220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:32.580243 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:32.580280 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:32.580294 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:32.580412 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.580789 (+ 377us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.580883 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:51:32.581975 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.571453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:40371 (local address 127.5.71.131:40735)
0923 20:51:32.572539 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.572584 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.574178 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.574742 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.574749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.574838 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:32.575409 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.575426 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.576626 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.576630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.577933 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.577944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.578760 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.578766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.578995 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.579772 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:32.579774 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:32.579889 (+ 115us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:32.579972 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:32.580417 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:32.580426 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:32.580432 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.580582 (+ 150us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.580887 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":391,"threads_started":1}
I20260923 20:51:32.583526 24699 heartbeater.cc:347] Connected to a master server at 127.5.71.190:40371
I20260923 20:51:32.583904 24699 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:32.584811 24699 heartbeater.cc:511] Master 127.5.71.190:40371 requested a full tablet report, sending...
I20260923 20:51:32.586340 24252 ts_manager.cc:194] Registered new tserver with Master: c79430ac0b954622b6a38310c3427e2f (127.5.71.131:34613)
I20260923 20:51:32.587212 24252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.131:40735
I20260923 20:51:32.599054 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:51:32.610062 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.603876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.5.71.190:40371)
0923 20:51:32.604058 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.604063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.604412 (+ 349us) server_negotiation.cc:407] Connection header received
0923 20:51:32.604565 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.604569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.604636 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.604736 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:32.605255 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.605962 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.607379 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.607850 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.607953 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.609222 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:32.609227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:32.609230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:32.609248 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:32.609281 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:32.609293 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:32.609424 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.609631 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.609680 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 20:51:32.614486 24252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:51:32.621452 24252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:51:32.647109 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.634915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:45295 (local address 127.0.0.1:48536)
0923 20:51:32.635837 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.635893 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.636096 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.638127 (+ 2031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.638136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.638165 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:32.638740 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.638765 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.641001 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.641007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.643965 (+ 2958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.643977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.646018 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.646023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.646246 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.646348 (+ 102us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.646483 (+ 135us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.646723 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":478,"spinlock_wait_cycles":768,"thread_start_us":361,"threads_started":1}
I20260923 20:51:32.647605 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.636273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.5.71.129:45639)
0923 20:51:32.637717 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.637724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.637749 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:51:32.637849 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.637855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.638094 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.639011 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:32.639224 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.641004 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.644302 (+ 3298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.645765 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.646682 (+ 917us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.646971 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.647045 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.647244 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1287,"thread_start_us":1182,"threads_started":1}
I20260923 20:51:32.648295 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.635643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45108)
0923 20:51:32.636763 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.636792 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.636912 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.638310 (+ 1398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.638314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.638347 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:32.638812 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.638822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.641218 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.641229 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.643717 (+ 2488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.643735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.646001 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.646009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.646253 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.646348 (+ 95us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.646470 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.647889 (+ 1419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":1215,"spinlock_wait_cycles":6400,"thread_start_us":2020,"threads_started":1}
I20260923 20:51:32.648509 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.635105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.5.71.130:45295)
0923 20:51:32.637408 (+ 2303us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.637426 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.637462 (+ 36us) server_negotiation.cc:407] Connection header received
0923 20:51:32.637575 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.637583 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.637845 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.638992 (+ 1147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:32.639233 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.640839 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.644177 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.645829 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.647596 (+ 1767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.647898 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.647976 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.648140 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2153,"thread_start_us":2015,"threads_started":1}
I20260923 20:51:32.652174 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.636545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34613 (local address 127.0.0.1:47430)
0923 20:51:32.644546 (+ 8001us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.644569 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.644700 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.645247 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.645251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.645275 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:32.645624 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.645637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.647519 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.647530 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.649197 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.649215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.651426 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.651433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.651598 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.651630 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.651733 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.651886 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7882,"thread_start_us":7738,"threads_started":1}
I20260923 20:51:32.652541 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.636922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47430 (local address 127.5.71.131:34613)
0923 20:51:32.644168 (+ 7246us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.644176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.644682 (+ 506us) server_negotiation.cc:407] Connection header received
0923 20:51:32.644859 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.644865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.645054 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.645269 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:32.645860 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.647356 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.649386 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.651221 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.651378 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.651624 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.651818 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.651982 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7040,"thread_start_us":6870,"threads_started":1}
I20260923 20:51:32.652444 24504 tablet_service.cc:1467] Processing CreateTablet for tablet 55f6b20168a549c9acd9d85f5294435f (DEFAULT_TABLE table=test-table [id=60e64e18e2fc44249c09f9c50792e0a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:32.652443 24373 tablet_service.cc:1467] Processing CreateTablet for tablet 55f6b20168a549c9acd9d85f5294435f (DEFAULT_TABLE table=test-table [id=60e64e18e2fc44249c09f9c50792e0a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:32.653990 24504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f6b20168a549c9acd9d85f5294435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:32.654440 24373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f6b20168a549c9acd9d85f5294435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:32.656312 24634 tablet_service.cc:1467] Processing CreateTablet for tablet 55f6b20168a549c9acd9d85f5294435f (DEFAULT_TABLE table=test-table [id=60e64e18e2fc44249c09f9c50792e0a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:32.657686 24634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55f6b20168a549c9acd9d85f5294435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:32.658526 24438 heartbeater.cc:503] Master 127.5.71.190:40371 was elected leader, sending a full tablet report...
I20260923 20:51:32.666687 24717 tablet_bootstrap.cc:492] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Bootstrap starting.
I20260923 20:51:32.670472 24717 tablet_bootstrap.cc:654] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:32.670993 24718 tablet_bootstrap.cc:492] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Bootstrap starting.
I20260923 20:51:32.672151 24717 log.cc:824] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:32.675477 24718 tablet_bootstrap.cc:654] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:32.676815 24720 tablet_bootstrap.cc:492] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Bootstrap starting.
I20260923 20:51:32.676935 24718 log.cc:824] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:32.678997 24717 tablet_bootstrap.cc:492] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: No bootstrap required, opened a new log
I20260923 20:51:32.679423 24717 ts_tablet_manager.cc:1397] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260923 20:51:32.680471 24720 tablet_bootstrap.cc:654] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:32.681829 24720 log.cc:824] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:32.682771 24718 tablet_bootstrap.cc:492] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: No bootstrap required, opened a new log
I20260923 20:51:32.683115 24718 ts_tablet_manager.cc:1397] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260923 20:51:32.686194 24720 tablet_bootstrap.cc:492] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: No bootstrap required, opened a new log
I20260923 20:51:32.686480 24720 ts_tablet_manager.cc:1397] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260923 20:51:32.688791 24717 raft_consensus.cc:348] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.689253 24717 raft_consensus.cc:374] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:32.689368 24717 raft_consensus.cc:729] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456240576e1e4ced976f7405698daf66, State: Initialized, Role: FOLLOWER
I20260923 20:51:32.690116 24717 consensus_queue.cc:260] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.692446 24717 ts_tablet_manager.cc:1428] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260923 20:51:32.695504 24718 raft_consensus.cc:348] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.696123 24718 raft_consensus.cc:374] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:32.696295 24718 raft_consensus.cc:729] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 211b5bac508f47e1b94477e16e8cbb99, State: Initialized, Role: FOLLOWER
I20260923 20:51:32.697091 24718 consensus_queue.cc:260] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.697284 24720 raft_consensus.cc:348] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.697749 24720 raft_consensus.cc:374] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:32.697862 24720 raft_consensus.cc:729] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79430ac0b954622b6a38310c3427e2f, State: Initialized, Role: FOLLOWER
I20260923 20:51:32.698612 24720 consensus_queue.cc:260] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.698936 24569 heartbeater.cc:503] Master 127.5.71.190:40371 was elected leader, sending a full tablet report...
I20260923 20:51:32.699400 24718 ts_tablet_manager.cc:1428] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260923 20:51:32.699880 24699 heartbeater.cc:503] Master 127.5.71.190:40371 was elected leader, sending a full tablet report...
I20260923 20:51:32.700528 24720 ts_tablet_manager.cc:1428] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260923 20:51:32.819973 24700 tablet.cc:2369] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:32.843840 24723 raft_consensus.cc:482] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:32.844192 24723 raft_consensus.cc:504] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.845997 24723 leader_election.cc:288] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 211b5bac508f47e1b94477e16e8cbb99 (127.5.71.130:45295), c79430ac0b954622b6a38310c3427e2f (127.5.71.131:34613)
W20260923 20:51:32.850431 24570 tablet.cc:2369] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:32.853228 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.846492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:40405 (local address 127.5.71.130:45295)
0923 20:51:32.846682 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.846686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.847238 (+ 552us) server_negotiation.cc:407] Connection header received
0923 20:51:32.847494 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.847504 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.847607 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.847675 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:32.848421 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.849135 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.851454 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.852268 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.852447 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.852518 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.852822 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.852951 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:51:32.853317 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.846408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:45295 (local address 127.5.71.129:40405)
0923 20:51:32.847094 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.847117 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.847277 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.847921 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.847928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.847950 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:32.848279 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.848287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.849298 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.849306 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.851220 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.851238 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.852460 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.852466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.852632 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.852681 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.852785 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.852942 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":5,"thread_start_us":353,"threads_started":1}
I20260923 20:51:32.853552 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.847242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34613 (local address 127.5.71.129:46737)
0923 20:51:32.847950 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:32.847965 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:32.848145 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:32.848508 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:32.848512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:32.848534 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:32.848851 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.848860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.849859 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.849865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.851401 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:32.851414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.852635 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.852640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:32.852874 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.852897 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 20:51:32.853002 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 20:51:32.853127 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":65,"thread_start_us":277,"threads_started":1}
I20260923 20:51:32.853848 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.847577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:46737 (local address 127.5.71.131:34613)
0923 20:51:32.847877 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.847884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.848064 (+ 180us) server_negotiation.cc:407] Connection header received
0923 20:51:32.848325 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.848329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.848386 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.848480 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:32.849009 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.849716 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.851601 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.852438 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.852622 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.852707 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.853033 (+ 326us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.853161 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:51:32.854367 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f6b20168a549c9acd9d85f5294435f" candidate_uuid: "456240576e1e4ced976f7405698daf66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "211b5bac508f47e1b94477e16e8cbb99" is_pre_election: true
I20260923 20:51:32.854748 24654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f6b20168a549c9acd9d85f5294435f" candidate_uuid: "456240576e1e4ced976f7405698daf66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79430ac0b954622b6a38310c3427e2f" is_pre_election: true
I20260923 20:51:32.854885 24524 raft_consensus.cc:2454] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456240576e1e4ced976f7405698daf66 in term 0.
I20260923 20:51:32.855192 24654 raft_consensus.cc:2454] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456240576e1e4ced976f7405698daf66 in term 0.
I20260923 20:51:32.855544 24328 leader_election.cc:302] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 211b5bac508f47e1b94477e16e8cbb99, 456240576e1e4ced976f7405698daf66; no voters:
I20260923 20:51:32.856004 24723 raft_consensus.cc:2781] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:51:32.856169 24723 raft_consensus.cc:482] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:51:32.856288 24723 raft_consensus.cc:3037] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:32.858482 24723 raft_consensus.cc:504] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.859105 24723 leader_election.cc:288] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [CANDIDATE]: Term 1 election: Requested vote from peers 211b5bac508f47e1b94477e16e8cbb99 (127.5.71.130:45295), c79430ac0b954622b6a38310c3427e2f (127.5.71.131:34613)
I20260923 20:51:32.859663 24524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f6b20168a549c9acd9d85f5294435f" candidate_uuid: "456240576e1e4ced976f7405698daf66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "211b5bac508f47e1b94477e16e8cbb99"
I20260923 20:51:32.859954 24524 raft_consensus.cc:3037] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:32.859968 24654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55f6b20168a549c9acd9d85f5294435f" candidate_uuid: "456240576e1e4ced976f7405698daf66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79430ac0b954622b6a38310c3427e2f"
I20260923 20:51:32.860201 24654 raft_consensus.cc:3037] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:32.862324 24524 raft_consensus.cc:2454] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456240576e1e4ced976f7405698daf66 in term 1.
I20260923 20:51:32.862385 24654 raft_consensus.cc:2454] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456240576e1e4ced976f7405698daf66 in term 1.
I20260923 20:51:32.862784 24328 leader_election.cc:302] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 211b5bac508f47e1b94477e16e8cbb99, 456240576e1e4ced976f7405698daf66; no voters:
I20260923 20:51:32.863096 24723 raft_consensus.cc:2781] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:32.863566 24723 raft_consensus.cc:686] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 1 LEADER]: Becoming Leader. State: Replica: 456240576e1e4ced976f7405698daf66, State: Running, Role: LEADER
I20260923 20:51:32.864154 24723 consensus_queue.cc:237] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } }
I20260923 20:51:32.869496 24251 catalog_manager.cc:5495] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 456240576e1e4ced976f7405698daf66 (127.5.71.129). New cstate: current_term: 1 leader_uuid: "456240576e1e4ced976f7405698daf66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456240576e1e4ced976f7405698daf66" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 45639 } health_report { overall_health: HEALTHY } } }
W20260923 20:51:32.883960 24439 tablet.cc:2369] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:32.893415 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.887476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.5.71.129:45639)
0923 20:51:32.887922 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.887928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.887949 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:51:32.888171 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.888175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.888279 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.888452 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:51:32.888838 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.889558 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.890998 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.891442 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.891596 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.892256 (+ 660us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:51:32.892795 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:51:32.893002 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.893120 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.893166 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260923 20:51:32.902997 24524 raft_consensus.cc:1261] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 1 FOLLOWER]: Refusing update from remote peer 456240576e1e4ced976f7405698daf66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:32.903059 24654 raft_consensus.cc:1261] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 1 FOLLOWER]: Refusing update from remote peer 456240576e1e4ced976f7405698daf66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:32.904068 24728 consensus_queue.cc:1035] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79430ac0b954622b6a38310c3427e2f" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:32.904461 24723 consensus_queue.cc:1035] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "211b5bac508f47e1b94477e16e8cbb99" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 45295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:32.916637 24731 mvcc.cc:205] Tried to move back new op lower bound from 7332645654119370752 to 7332645653975748608. Current Snapshot: MvccSnapshot[applied={T|T < 7332645654119370752}]
I20260923 20:51:32.932881 24251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:32.933240 24251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:32.937286 24251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 576a90e8b4394284a9bc1611b162c138: Sending DeleteTablet for 3 replicas of tablet 55f6b20168a549c9acd9d85f5294435f
I20260923 20:51:32.938689 24504 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f6b20168a549c9acd9d85f5294435f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:32 UTC) from {username='slave'} at 127.0.0.1:48536
I20260923 20:51:32.938689 24634 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f6b20168a549c9acd9d85f5294435f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:32 UTC) from {username='slave'} at 127.0.0.1:47430
I20260923 20:51:32.938748 24373 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f6b20168a549c9acd9d85f5294435f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:32 UTC) from {username='slave'} at 127.0.0.1:45108
I20260923 20:51:32.940918 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24311
I20260923 20:51:32.950580 24742 tablet_replica.cc:330] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: stopping tablet replica
I20260923 20:51:32.951397 24742 raft_consensus.cc:2229] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:51:32.952308 24742 raft_consensus.cc:2258] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:32.952972 24741 tablet_replica.cc:330] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: stopping tablet replica
I20260923 20:51:32.953970 24740 tablet_replica.cc:330] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: stopping tablet replica
I20260923 20:51:32.954417 24740 raft_consensus.cc:2229] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:32.954620 24741 raft_consensus.cc:2229] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:32.954939 24740 raft_consensus.cc:2258] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:32.955089 24741 raft_consensus.cc:2258] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:32.957963 24740 ts_tablet_manager.cc:1905] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:32.958047 24741 ts_tablet_manager.cc:1905] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:32.961366 24742 ts_tablet_manager.cc:1905] T 55f6b20168a549c9acd9d85f5294435f P 456240576e1e4ced976f7405698daf66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:32.962572 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:32.943002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.5.71.129:45639)
0923 20:51:32.943196 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:32.943207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:32.953279 (+ 10072us) server_negotiation.cc:407] Connection header received
0923 20:51:32.953509 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:32.953515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:32.953615 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:32.953734 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:32.954395 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.955571 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.957620 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:32.958335 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:32.958476 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:32.960072 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:32.960085 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:32.960104 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:32.960935 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:32.961001 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:32.961025 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:32.961133 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:32.962053 (+ 920us) server_negotiation.cc:299] Negotiation successful
0923 20:51:32.962177 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 20:51:32.963454 24433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:32.968472 24740 ts_tablet_manager.cc:1918] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:32.968588 24741 ts_tablet_manager.cc:1918] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:32.968685 24740 log.cc:1196] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/55f6b20168a549c9acd9d85f5294435f
I20260923 20:51:32.968761 24741 log.cc:1196] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/55f6b20168a549c9acd9d85f5294435f
I20260923 20:51:32.969183 24740 ts_tablet_manager.cc:1939] T 55f6b20168a549c9acd9d85f5294435f P 211b5bac508f47e1b94477e16e8cbb99: Deleting consensus metadata
I20260923 20:51:32.969280 24741 ts_tablet_manager.cc:1939] T 55f6b20168a549c9acd9d85f5294435f P c79430ac0b954622b6a38310c3427e2f: Deleting consensus metadata
I20260923 20:51:32.971016 24238 catalog_manager.cc:4841] TS 211b5bac508f47e1b94477e16e8cbb99 (127.5.71.130:45295): tablet 55f6b20168a549c9acd9d85f5294435f (table test-table [id=60e64e18e2fc44249c09f9c50792e0a8]) successfully deleted
I20260923 20:51:32.971405 24238 catalog_manager.cc:4841] TS c79430ac0b954622b6a38310c3427e2f (127.5.71.131:34613): tablet 55f6b20168a549c9acd9d85f5294435f (table test-table [id=60e64e18e2fc44249c09f9c50792e0a8]) successfully deleted
I20260923 20:51:33.073931 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24311
W20260923 20:51:33.093317 24239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:45639 (error 108)
I20260923 20:51:33.093529 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24442
I20260923 20:51:33.095683 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.095022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45122)
0923 20:51:33.095305 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.095456 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260923 20:51:33.096230 24239 catalog_manager.cc:4584] TS 456240576e1e4ced976f7405698daf66 (127.5.71.129:45639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55f6b20168a549c9acd9d85f5294435f: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
I20260923 20:51:33.102694 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.094480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.5.71.130:45295)
0923 20:51:33.094701 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:33.094710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:33.095172 (+ 462us) server_negotiation.cc:407] Connection header received
0923 20:51:33.095347 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:33.095352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:33.095477 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:33.095600 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:33.096147 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:33.097158 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:33.099033 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:33.099474 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:33.099574 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:33.100961 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:33.100970 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:33.100984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:33.101685 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:33.101746 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:33.101767 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:33.101856 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:33.102186 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 20:51:33.102319 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:51:33.103381 24564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:33.150280 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.149640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45136)
0923 20:51:33.149978 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.150104 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260923 20:51:33.151430 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.151039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45146)
0923 20:51:33.151194 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.151308 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260923 20:51:33.197059 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24442
I20260923 20:51:33.212054 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.211504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45148)
0923 20:51:33.211747 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.211878 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260923 20:51:33.213143 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.212783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45158)
0923 20:51:33.212956 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.213020 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260923 20:51:33.216238 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24572
I20260923 20:51:33.224277 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.217170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.5.71.131:34613)
0923 20:51:33.217446 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:33.217453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:33.217572 (+ 119us) server_negotiation.cc:407] Connection header received
0923 20:51:33.217898 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:33.217902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:33.217994 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:33.218188 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:33.218608 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:33.219425 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:33.220893 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:33.221331 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:33.221466 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:33.222826 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:33.222837 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:33.222854 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:33.223484 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:33.223534 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:33.223555 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:33.223649 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:33.223907 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 20:51:33.224010 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260923 20:51:33.225106 24694 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:33.294072 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.293540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45168)
0923 20:51:33.293787 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.293884 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260923 20:51:33.295349 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:33.294921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:45639 (local address 127.0.0.1:45174)
0923 20:51:33.295148 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:33.295228 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:45639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260923 20:51:33.318835 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24572
I20260923 20:51:33.336427 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24221
I20260923 20:51:33.337641 24282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:33.456293 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24221
2026-09-23T20:51:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3037 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:33 dist-test-slave-dn29 krb5kdc[24751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:33 dist-test-slave-dn29 krb5kdc[24751](info): set up 2 sockets
Sep 23 20:51:33 dist-test-slave-dn29 krb5kdc[24751](info): commencing operation
krb5kdc: starting...
W20260923 20:51:34.668395 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.011s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:34 dist-test-slave-dn29 krb5kdc[24751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:51:34.948755 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38981
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:35.306136 24764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:35.306473 24764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:35.306568 24764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:35.314713 24764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:35.314838 24764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:35.314885 24764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:35.314936 24764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:35.314980 24764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:35.325759 24764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:38981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:35.327150 24764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:35.328573 24764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:35.341226 24770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:35.341177 24769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:35.341192 24772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:35.342798 24764 server_base.cc:1034] running on GCE node
I20260923 20:51:35.343648 24764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:35.345242 24764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:35.346648 24764 hybrid_clock.cc:648] HybridClock initialized: now 1790196695346594 us; error 41 us; skew 500 ppm
I20260923 20:51:35.347075 24764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:35 dist-test-slave-dn29 krb5kdc[24751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:35.358309 24764 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:51:35.360317 24764 webserver.cc:460] Webserver started at http://127.5.71.190:46387/ using document root <none> and password file <none>
I20260923 20:51:35.361107 24764 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:35.361223 24764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:35.361595 24764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:35.364888 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d"
format_stamp: "Formatted at 2026-09-23 20:51:35 on dist-test-slave-dn29"
server_key: "f23c28dc93a7e5f74d7131ac78759abd"
server_key_iv: "0d987adda17d2b1b7828472647bd28ed"
server_key_version: "encryptionkey@0"
I20260923 20:51:35.365746 24764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d"
format_stamp: "Formatted at 2026-09-23 20:51:35 on dist-test-slave-dn29"
server_key: "f23c28dc93a7e5f74d7131ac78759abd"
server_key_iv: "0d987adda17d2b1b7828472647bd28ed"
server_key_version: "encryptionkey@0"
I20260923 20:51:35.371520 24764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260923 20:51:35.375576 24779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:35.377223 24764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:35.377434 24764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d"
format_stamp: "Formatted at 2026-09-23 20:51:35 on dist-test-slave-dn29"
server_key: "f23c28dc93a7e5f74d7131ac78759abd"
server_key_iv: "0d987adda17d2b1b7828472647bd28ed"
server_key_version: "encryptionkey@0"
I20260923 20:51:35.377663 24764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:35.392604 24764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:35.397240 24764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:35.397574 24764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:35.417866 24764 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:38981
I20260923 20:51:35.417980 24830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:38981 every 8 connection(s)
I20260923 20:51:35.419375 24764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:35.423475 24831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:35.427115 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24764
I20260923 20:51:35.427316 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:35.427897 5406 external_mini_cluster.cc:1428] Setting key d81602f6b98dcfdd675b1b86525fb097
I20260923 20:51:35.437512 24831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: Bootstrap starting.
I20260923 20:51:35.442494 24831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:35.444036 24831 log.cc:824] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:51:35 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:35.448001 24831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: No bootstrap required, opened a new log
I20260923 20:51:35.455106 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:35.430084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.5.71.190:38981)
0923 20:51:35.430989 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:35.431001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:35.431044 (+ 43us) server_negotiation.cc:407] Connection header received
0923 20:51:35.432931 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:35.432957 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:35.433407 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:35.433789 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:35.435955 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:35.437311 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:35.439216 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:35.440503 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:35.440875 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:35.446255 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:35.446265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:35.446285 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:35.446324 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:35.450117 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:35.451162 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:35.451170 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:35.451178 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:35.451415 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:35.451922 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:35.451926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:35.451928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:35.452568 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:35.452815 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:35.453371 (+ 556us) server_negotiation.cc:299] Negotiation successful
0923 20:51:35.453947 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":419,"threads_started":1}
I20260923 20:51:35.457472 24831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38981 } }
I20260923 20:51:35.457916 24831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:35.458086 24831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f15cd71e3c48c3a3e4cbed3abfc18d, State: Initialized, Role: FOLLOWER
I20260923 20:51:35.458669 24831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38981 } }
I20260923 20:51:35.458940 24831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:35.459106 24831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:35.459280 24831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:35.461689 24831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38981 } }
I20260923 20:51:35.462265 24831 leader_election.cc:302] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f15cd71e3c48c3a3e4cbed3abfc18d; no voters:
I20260923 20:51:35.463260 24831 leader_election.cc:288] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:35.463315 24836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:35.464871 24836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [term 1 LEADER]: Becoming Leader. State: Replica: 53f15cd71e3c48c3a3e4cbed3abfc18d, State: Running, Role: LEADER
I20260923 20:51:35.465472 24836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38981 } }
I20260923 20:51:35.465867 24831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:35.469931 24837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f15cd71e3c48c3a3e4cbed3abfc18d. Latest consensus state: current_term: 1 leader_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38981 } } }
I20260923 20:51:35.470880 24837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:35.471239 24838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f15cd71e3c48c3a3e4cbed3abfc18d" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 38981 } } }
I20260923 20:51:35.471626 24838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d [sys.catalog]: This master's current role is: LEADER
W20260923 20:51:35.479724 24852 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:51:35.479984 24852 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:51:35.480172 24846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:35.482236 24846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:35.491940 24846 catalog_manager.cc:1347] Generated new cluster ID: f23007a123314b9ab8653bf6f9e11b1a
I20260923 20:51:35.492089 24846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:35.517078 24846 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:35.518585 24846 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:35.532570 24846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: Generated new TSK 0
I20260923 20:51:35.533639 24846 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.71.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
Entry for principal kudu/127.5.71.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab.
I20260923 20:51:35.608017 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38981
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:35.956480 24857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:35.956795 24857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:35.956959 24857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:35.964617 24857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:35.964753 24857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:35.964921 24857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:35.975538 24857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.129.keytab
--principal=kudu/127.5.71.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:35.977044 24857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:35.978475 24857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:35.993243 24862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:35.993173 24863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:35.993273 24865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:35.993921 24857 server_base.cc:1034] running on GCE node
I20260923 20:51:35.994594 24857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:35.995680 24857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:35.996939 24857 hybrid_clock.cc:648] HybridClock initialized: now 1790196695996883 us; error 47 us; skew 500 ppm
I20260923 20:51:35.997330 24857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:35 dist-test-slave-dn29 krb5kdc[24751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:36.001246 24857 init.cc:377] Logged in from keytab as kudu/127.5.71.129@KRBTEST.COM (short username kudu)
I20260923 20:51:36.002995 24857 webserver.cc:460] Webserver started at http://127.5.71.129:41979/ using document root <none> and password file <none>
I20260923 20:51:36.003895 24857 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:36.004055 24857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:36.004392 24857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:36.007660 24857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "32f777c8850048cc8aaffa69109648f5"
format_stamp: "Formatted at 2026-09-23 20:51:36 on dist-test-slave-dn29"
server_key: "d306b4f240ea2fb85634f24a57d8561b"
server_key_iv: "ecc275e256b5e4d194b9cb900d390890"
server_key_version: "encryptionkey@0"
I20260923 20:51:36.008793 24857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "32f777c8850048cc8aaffa69109648f5"
format_stamp: "Formatted at 2026-09-23 20:51:36 on dist-test-slave-dn29"
server_key: "d306b4f240ea2fb85634f24a57d8561b"
server_key_iv: "ecc275e256b5e4d194b9cb900d390890"
server_key_version: "encryptionkey@0"
I20260923 20:51:36.014532 24857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:51:36.018764 24872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:36.020421 24857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:36.020613 24857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "32f777c8850048cc8aaffa69109648f5"
format_stamp: "Formatted at 2026-09-23 20:51:36 on dist-test-slave-dn29"
server_key: "d306b4f240ea2fb85634f24a57d8561b"
server_key_iv: "ecc275e256b5e4d194b9cb900d390890"
server_key_version: "encryptionkey@0"
I20260923 20:51:36.020841 24857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:36.054695 24857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:36.059917 24857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:36.060267 24857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:36.061338 24857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:36.062765 24857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:36.062879 24857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:36.063040 24857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:36.063115 24857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:36.102774 24857 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:38909
I20260923 20:51:36.102878 24984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:38909 every 8 connection(s)
I20260923 20:51:36.104226 24857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:36.108682 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24857
I20260923 20:51:36.108865 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:51:36.109321 5406 external_mini_cluster.cc:1428] Setting key f92c9ed86ac005927c1ed8607df27c31
Sep 23 20:51:36 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196695, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.129@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:36.129366 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:36.106972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:60875 (local address 127.5.71.190:38981)
0923 20:51:36.107446 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:36.107453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:36.108700 (+ 1247us) server_negotiation.cc:407] Connection header received
0923 20:51:36.111140 (+ 2440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:36.111147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:36.111243 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:36.111380 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:36.113111 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.114466 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.116361 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.117182 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.117346 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:36.121333 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:36.121344 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:36.121351 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:36.121401 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:36.125348 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:36.126364 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:36.126373 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:36.126379 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:36.126549 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:36.127090 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:36.127097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:36.127103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:36.127494 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:36.127660 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:36.128572 (+ 912us) server_negotiation.cc:299] Negotiation successful
0923 20:51:36.128859 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":233,"threads_started":1}
I20260923 20:51:36.130067 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:36.107550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38981 (local address 127.5.71.129:60875)
0923 20:51:36.108433 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:36.108475 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:36.110787 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:36.111673 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:36.111682 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:36.112303 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:36.112916 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:36.112936 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.114667 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.114704 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:36.116201 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:36.116215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.117438 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.117444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:36.117671 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:36.118444 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:36.118473 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:36.121053 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:36.125588 (+ 4535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:36.125601 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:36.125621 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:36.126212 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:36.126733 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:36.126736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:36.126739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:36.126954 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:36.127663 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:36.127670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:36.128140 (+ 470us) client_negotiation.cc:769] Sending connection context
0923 20:51:36.128416 (+ 276us) client_negotiation.cc:240] Negotiation successful
0923 20:51:36.128863 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":352,"threads_started":1}
I20260923 20:51:36.131980 24985 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38981
I20260923 20:51:36.132319 24985 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:36.133098 24985 heartbeater.cc:511] Master 127.5.71.190:38981 requested a full tablet report, sending...
I20260923 20:51:36.136021 24796 ts_manager.cc:194] Registered new tserver with Master: 32f777c8850048cc8aaffa69109648f5 (127.5.71.129:38909)
WARNING: no policy specified for kudu/127.5.71.130@KRBTEST.COM; defaulting to no policy
I20260923 20:51:36.137697 24796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.129@KRBTEST.COM'} at 127.5.71.129:60875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
Entry for principal kudu/127.5.71.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab.
I20260923 20:51:36.171928 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38981
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:36.528487 24991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:36.528872 24991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:36.529062 24991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:36.537382 24991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:36.537523 24991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:36.537669 24991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:36.548111 24991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.130.keytab
--principal=kudu/127.5.71.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:36.549383 24991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:36.550702 24991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:36.565199 24999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:36.565249 24997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:36.565269 24996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:36.573998 24991 server_base.cc:1034] running on GCE node
I20260923 20:51:36.574893 24991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:36.576098 24991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:36.577386 24991 hybrid_clock.cc:648] HybridClock initialized: now 1790196696577348 us; error 43 us; skew 500 ppm
I20260923 20:51:36.577802 24991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:36 dist-test-slave-dn29 krb5kdc[24751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196696, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:36.581450 24991 init.cc:377] Logged in from keytab as kudu/127.5.71.130@KRBTEST.COM (short username kudu)
I20260923 20:51:36.583354 24991 webserver.cc:460] Webserver started at http://127.5.71.130:41597/ using document root <none> and password file <none>
I20260923 20:51:36.584156 24991 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:36.584275 24991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:36.584553 24991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:36.587551 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "526ecd5444064266a654322c8cb21962"
format_stamp: "Formatted at 2026-09-23 20:51:36 on dist-test-slave-dn29"
server_key: "54497ba2afc843672a1bf3bd446af383"
server_key_iv: "7dba8a1424ee0fdd451d8ded4e51f7f1"
server_key_version: "encryptionkey@0"
I20260923 20:51:36.588579 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "526ecd5444064266a654322c8cb21962"
format_stamp: "Formatted at 2026-09-23 20:51:36 on dist-test-slave-dn29"
server_key: "54497ba2afc843672a1bf3bd446af383"
server_key_iv: "7dba8a1424ee0fdd451d8ded4e51f7f1"
server_key_version: "encryptionkey@0"
I20260923 20:51:36.594262 24991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:51:36.598238 25006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:36.599871 24991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:51:36.600060 24991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "526ecd5444064266a654322c8cb21962"
format_stamp: "Formatted at 2026-09-23 20:51:36 on dist-test-slave-dn29"
server_key: "54497ba2afc843672a1bf3bd446af383"
server_key_iv: "7dba8a1424ee0fdd451d8ded4e51f7f1"
server_key_version: "encryptionkey@0"
I20260923 20:51:36.600291 24991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:36.623322 24991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:36.628723 24991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:36.629055 24991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:36.630195 24991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:36.631942 24991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:36.632030 24991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:36.632213 24991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:36.632266 24991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:36.667587 24991 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:37499
I20260923 20:51:36.667678 25118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:37499 every 8 connection(s)
I20260923 20:51:36.668970 24991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:36.670135 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 24991
I20260923 20:51:36.670352 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:51:36.670944 5406 external_mini_cluster.cc:1428] Setting key 7e63518885e2694d0031d9976e40d9a9
Sep 23 20:51:36 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196696, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.130@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:36.692186 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:36.672345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:38249 (local address 127.5.71.190:38981)
0923 20:51:36.672971 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:36.672977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:36.673984 (+ 1007us) server_negotiation.cc:407] Connection header received
0923 20:51:36.675956 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:36.675962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:36.676051 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:36.676191 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:36.677747 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.678481 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.680131 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.680631 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.680776 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:36.685410 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:36.685417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:36.685420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:36.685459 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:36.688480 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:36.689430 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:36.689436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:36.689443 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:36.689576 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:36.690071 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:36.690084 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:36.690087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:36.690394 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:36.690582 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:36.691451 (+ 869us) server_negotiation.cc:299] Negotiation successful
0923 20:51:36.691724 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":359,"threads_started":1}
I20260923 20:51:36.693298 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:36.672952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38981 (local address 127.5.71.130:38249)
0923 20:51:36.673835 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:36.673878 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:36.675622 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:36.676437 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:36.676446 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:36.676972 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:36.677568 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:36.677590 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.678641 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.678653 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:36.679990 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:36.680001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:36.680873 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:36.680881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:36.681221 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:36.682028 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:36.682068 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:36.685073 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:36.688696 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:36.688708 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:36.688729 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:36.689275 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:36.689708 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:36.689714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:36.689718 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:36.689938 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:36.690551 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:36.690561 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:36.690970 (+ 409us) client_negotiation.cc:769] Sending connection context
0923 20:51:36.691323 (+ 353us) client_negotiation.cc:240] Negotiation successful
0923 20:51:36.691972 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":390,"threads_started":1}
I20260923 20:51:36.694941 25119 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38981
I20260923 20:51:36.695312 25119 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:36.696311 25119 heartbeater.cc:511] Master 127.5.71.190:38981 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.71.131@KRBTEST.COM; defaulting to no policy
I20260923 20:51:36.698289 24796 ts_manager.cc:194] Registered new tserver with Master: 526ecd5444064266a654322c8cb21962 (127.5.71.130:37499)
I20260923 20:51:36.699327 24796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.130@KRBTEST.COM'} at 127.5.71.130:38249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
Entry for principal kudu/127.5.71.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab.
I20260923 20:51:36.732249 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38981
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:37.080214 25125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:37.080578 25125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:37.080755 25125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:37.088824 25125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:37.088991 25125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:37.089169 25125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:37.099421 25125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.131.keytab
--principal=kudu/127.5.71.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:37.100914 25125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:37.102391 25125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:37.116384 25130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:37.116345 25133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:37.116431 25131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:37.121977 25125 server_base.cc:1034] running on GCE node
I20260923 20:51:37.122816 25125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:37.123999 25125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:37.125259 25125 hybrid_clock.cc:648] HybridClock initialized: now 1790196697125208 us; error 45 us; skew 500 ppm
I20260923 20:51:37.125651 25125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:37 dist-test-slave-dn29 krb5kdc[24751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196697, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:37.129155 25125 init.cc:377] Logged in from keytab as kudu/127.5.71.131@KRBTEST.COM (short username kudu)
I20260923 20:51:37.130829 25125 webserver.cc:460] Webserver started at http://127.5.71.131:36573/ using document root <none> and password file <none>
I20260923 20:51:37.131603 25125 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:37.131762 25125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:37.132129 25125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:37.135288 25125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "a1e9b62805034d94bcaab7cc632a1864"
format_stamp: "Formatted at 2026-09-23 20:51:37 on dist-test-slave-dn29"
server_key: "71eef2f5dfde67f448e00d3733c45479"
server_key_iv: "272c94f6e986beda8c74bd7755976d6a"
server_key_version: "encryptionkey@0"
I20260923 20:51:37.136210 25125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "a1e9b62805034d94bcaab7cc632a1864"
format_stamp: "Formatted at 2026-09-23 20:51:37 on dist-test-slave-dn29"
server_key: "71eef2f5dfde67f448e00d3733c45479"
server_key_iv: "272c94f6e986beda8c74bd7755976d6a"
server_key_version: "encryptionkey@0"
I20260923 20:51:37.141629 24985 heartbeater.cc:503] Master 127.5.71.190:38981 was elected leader, sending a full tablet report...
I20260923 20:51:37.142578 25125 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:51:37.146657 25140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:37.148409 25125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:37.148692 25125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "a1e9b62805034d94bcaab7cc632a1864"
format_stamp: "Formatted at 2026-09-23 20:51:37 on dist-test-slave-dn29"
server_key: "71eef2f5dfde67f448e00d3733c45479"
server_key_iv: "272c94f6e986beda8c74bd7755976d6a"
server_key_version: "encryptionkey@0"
I20260923 20:51:37.148964 25125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:37.170543 25125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:37.175963 25125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:37.176280 25125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:37.177352 25125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:37.178788 25125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:37.178895 25125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:37.179055 25125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:37.179127 25125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:37.216042 25125 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:42907
I20260923 20:51:37.216135 25252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:42907 every 8 connection(s)
I20260923 20:51:37.217409 25125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:37.219010 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25125
I20260923 20:51:37.219195 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:51:37.219720 5406 external_mini_cluster.cc:1428] Setting key 5bc4d8dff5f44dde62ca271d19ee7e53
Sep 23 20:51:37 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196697, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.131@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:51:37.241407 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.221500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:46509 (local address 127.5.71.190:38981)
0923 20:51:37.222044 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.222052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.223644 (+ 1592us) server_negotiation.cc:407] Connection header received
0923 20:51:37.225709 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.225715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.225837 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.225983 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:37.227686 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.228667 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.230332 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.230784 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.230894 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.235303 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:37.235310 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:37.235318 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:37.235384 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:37.238330 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.239077 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.239097 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.239099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.239254 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.239728 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.239731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.239733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.240107 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:37.240227 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.240926 (+ 699us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.241129 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":255,"threads_started":1}
I20260923 20:51:37.242393 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.222278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:38981 (local address 127.5.71.131:46509)
0923 20:51:37.223485 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:37.223528 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:37.225419 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:37.226266 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:37.226276 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:37.226858 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:37.227487 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.227507 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.228845 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.228849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.230140 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.230152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.230998 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.231003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.231239 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.232167 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:51:37.232200 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:37.234999 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:37.238470 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:37.238477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:37.238492 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:37.238964 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:37.239409 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:51:37.239413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:51:37.239415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:51:37.239621 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:51:37.240251 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:37.240259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:37.240592 (+ 333us) client_negotiation.cc:769] Sending connection context
0923 20:51:37.240838 (+ 246us) client_negotiation.cc:240] Negotiation successful
0923 20:51:37.241249 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":630,"threads_started":1}
I20260923 20:51:37.244109 25253 heartbeater.cc:347] Connected to a master server at 127.5.71.190:38981
I20260923 20:51:37.244447 25253 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:37.245288 25253 heartbeater.cc:511] Master 127.5.71.190:38981 requested a full tablet report, sending...
I20260923 20:51:37.246909 24796 ts_manager.cc:194] Registered new tserver with Master: a1e9b62805034d94bcaab7cc632a1864 (127.5.71.131:42907)
I20260923 20:51:37.248147 24796 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.71.131@KRBTEST.COM'} at 127.5.71.131:46509
I20260923 20:51:37.249332 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:51:37.268006 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.253872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.5.71.190:38981)
0923 20:51:37.254102 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.254112 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.254600 (+ 488us) server_negotiation.cc:407] Connection header received
0923 20:51:37.254802 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.254806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.254881 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.254977 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:37.256440 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.257191 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.258587 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.259013 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.259118 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.261745 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:37.261750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:37.261753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:37.261788 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:37.265057 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.265981 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.265985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.265988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.266166 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.266525 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.266528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.266530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.267019 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:37.267254 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.267544 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.267732 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 20:51:37.272886 24795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:51:37.280820 24795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:51:37.307701 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.295530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.5.71.129:38909)
0923 20:51:37.298264 (+ 2734us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.298276 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.298310 (+ 34us) server_negotiation.cc:407] Connection header received
0923 20:51:37.298426 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.298437 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.298731 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.299530 (+ 799us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:37.299940 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.301606 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.304749 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.306385 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.306545 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.306810 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.307131 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.307316 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2535,"thread_start_us":2365,"threads_started":1}
I20260923 20:51:37.307780 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.295159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:38909 (local address 127.0.0.1:59684)
0923 20:51:37.297501 (+ 2342us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:37.297527 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:37.297646 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:37.299006 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:37.299016 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:37.299052 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:37.299647 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.299662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.301774 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.301780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.304585 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.304598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.306618 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.306630 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.306857 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.306926 (+ 69us) client_negotiation.cc:769] Sending connection context
0923 20:51:37.307065 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 20:51:37.307330 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2192,"mutex_wait_us":40,"thread_start_us":1878,"threads_started":1}
I20260923 20:51:37.307911 25266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.294583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:42907 (local address 127.0.0.1:34458)
0923 20:51:37.296103 (+ 1520us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:37.296173 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:37.296393 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:37.299252 (+ 2859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:37.299259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:37.299291 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:37.300101 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.300121 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.302258 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.302270 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.304403 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.304419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.306840 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.306847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.307068 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.307103 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 20:51:37.307229 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 20:51:37.307379 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":122,"thread_start_us":1045,"threads_started":1}
I20260923 20:51:37.308866 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.294971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.5.71.131:42907)
0923 20:51:37.298362 (+ 3391us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.298369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.298398 (+ 29us) server_negotiation.cc:407] Connection header received
0923 20:51:37.298481 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.298488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.298730 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.298945 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:37.300324 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.302108 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.304623 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.306258 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.306359 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.306619 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.308319 (+ 1700us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.308494 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1973,"thread_start_us":1816,"threads_started":1}
I20260923 20:51:37.312348 24920 tablet_service.cc:1467] Processing CreateTablet for tablet 1e7aae62b40f4320b22d65ccfb050ca4 (DEFAULT_TABLE table=test-table [id=7cebd832829141f49a7884dc5784d288]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:37.312556 25188 tablet_service.cc:1467] Processing CreateTablet for tablet 1e7aae62b40f4320b22d65ccfb050ca4 (DEFAULT_TABLE table=test-table [id=7cebd832829141f49a7884dc5784d288]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:37.313722 24920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e7aae62b40f4320b22d65ccfb050ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:37.313862 25188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e7aae62b40f4320b22d65ccfb050ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:37.317921 25270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.297728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:37499 (local address 127.0.0.1:51984)
0923 20:51:37.300349 (+ 2621us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:37.300376 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:37.300506 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:37.301391 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:37.301395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:37.301423 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:37.301910 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.301923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.304159 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.304165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.306659 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.306676 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.316844 (+ 10168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.316853 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.317101 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.317145 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 20:51:37.317268 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 20:51:37.317446 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2493,"mutex_wait_us":3,"thread_start_us":2292,"threads_started":1}
I20260923 20:51:37.318903 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.298285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.5.71.130:37499)
0923 20:51:37.298951 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.298964 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.300502 (+ 1538us) server_negotiation.cc:407] Connection header received
0923 20:51:37.300651 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.300661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.300955 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.301143 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:37.302137 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.304004 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.306860 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.308845 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.309014 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.309255 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.318295 (+ 9040us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.318521 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":322,"threads_started":1}
I20260923 20:51:37.322361 25054 tablet_service.cc:1467] Processing CreateTablet for tablet 1e7aae62b40f4320b22d65ccfb050ca4 (DEFAULT_TABLE table=test-table [id=7cebd832829141f49a7884dc5784d288]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:37.323844 25054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e7aae62b40f4320b22d65ccfb050ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:37.327777 25272 tablet_bootstrap.cc:492] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Bootstrap starting.
I20260923 20:51:37.331826 25272 tablet_bootstrap.cc:654] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:37.332103 25273 tablet_bootstrap.cc:492] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Bootstrap starting.
I20260923 20:51:37.334535 25272 log.cc:824] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:37.335996 25273 tablet_bootstrap.cc:654] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:37.337093 25273 log.cc:824] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:37.337917 25272 tablet_bootstrap.cc:492] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: No bootstrap required, opened a new log
I20260923 20:51:37.338212 25272 ts_tablet_manager.cc:1397] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260923 20:51:37.339679 25276 tablet_bootstrap.cc:492] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Bootstrap starting.
I20260923 20:51:37.340498 25273 tablet_bootstrap.cc:492] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: No bootstrap required, opened a new log
I20260923 20:51:37.340791 25273 ts_tablet_manager.cc:1397] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260923 20:51:37.344048 25276 tablet_bootstrap.cc:654] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:37.345526 25276 log.cc:824] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:37.346796 25272 raft_consensus.cc:348] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.347312 25272 raft_consensus.cc:374] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:37.347460 25272 raft_consensus.cc:729] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e9b62805034d94bcaab7cc632a1864, State: Initialized, Role: FOLLOWER
I20260923 20:51:37.348182 25272 consensus_queue.cc:260] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.349221 25276 tablet_bootstrap.cc:492] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: No bootstrap required, opened a new log
I20260923 20:51:37.349324 25253 heartbeater.cc:503] Master 127.5.71.190:38981 was elected leader, sending a full tablet report...
I20260923 20:51:37.349561 25276 ts_tablet_manager.cc:1397] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260923 20:51:37.350075 25272 ts_tablet_manager.cc:1428] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260923 20:51:37.352511 25273 raft_consensus.cc:348] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.353098 25273 raft_consensus.cc:374] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:37.353227 25273 raft_consensus.cc:729] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32f777c8850048cc8aaffa69109648f5, State: Initialized, Role: FOLLOWER
I20260923 20:51:37.354034 25273 consensus_queue.cc:260] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.356230 25273 ts_tablet_manager.cc:1428] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
W20260923 20:51:37.356932 24986 tablet.cc:2369] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:37.361462 25276 raft_consensus.cc:348] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.362046 25276 raft_consensus.cc:374] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:37.362187 25276 raft_consensus.cc:729] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 526ecd5444064266a654322c8cb21962, State: Initialized, Role: FOLLOWER
I20260923 20:51:37.362970 25276 consensus_queue.cc:260] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.364435 25119 heartbeater.cc:503] Master 127.5.71.190:38981 was elected leader, sending a full tablet report...
I20260923 20:51:37.365046 25276 ts_tablet_manager.cc:1428] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
W20260923 20:51:37.421384 25120 tablet.cc:2369] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:51:37.469157 25254 tablet.cc:2369] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:37.515100 25278 raft_consensus.cc:482] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:37.515477 25278 raft_consensus.cc:504] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.517520 25278 leader_election.cc:288] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32f777c8850048cc8aaffa69109648f5 (127.5.71.129:38909), 526ecd5444064266a654322c8cb21962 (127.5.71.130:37499)
I20260923 20:51:37.525261 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.517973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:38909 (local address 127.5.71.131:50613)
0923 20:51:37.518261 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:37.518290 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:37.518455 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:37.519013 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:37.519017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:37.519044 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:37.519300 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.519312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.520830 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.520833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.522840 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.522855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.524596 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.524605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.524774 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.524801 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 20:51:37.524883 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 20:51:37.524982 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260923 20:51:37.525364 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.518158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:50613 (local address 127.5.71.129:38909)
0923 20:51:37.518399 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.518410 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.518433 (+ 23us) server_negotiation.cc:407] Connection header received
0923 20:51:37.518623 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.518632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.518714 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.518821 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:37.519462 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.520670 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.523033 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.524400 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.524606 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.524693 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.524931 (+ 238us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.525090 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 20:51:37.526216 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.518393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:37499 (local address 127.5.71.131:37349)
0923 20:51:37.519143 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:37.519160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:37.519285 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:37.519690 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:37.519693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:37.519712 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:37.520061 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.520069 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.521386 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.521391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.523279 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:37.523290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.525370 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.525377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:37.525581 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.525614 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 20:51:37.525762 (+ 148us) client_negotiation.cc:240] Negotiation successful
0923 20:51:37.525906 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":412,"threads_started":1}
I20260923 20:51:37.526233 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.518764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:37349 (local address 127.5.71.130:37499)
0923 20:51:37.518973 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.518979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.519279 (+ 300us) server_negotiation.cc:407] Connection header received
0923 20:51:37.519454 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.519460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.519553 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.519672 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:37.520218 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.521225 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.523454 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.525077 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.525253 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.525324 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.525816 (+ 492us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.525972 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 20:51:37.526744 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7aae62b40f4320b22d65ccfb050ca4" candidate_uuid: "a1e9b62805034d94bcaab7cc632a1864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32f777c8850048cc8aaffa69109648f5" is_pre_election: true
I20260923 20:51:37.527370 24940 raft_consensus.cc:2454] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e9b62805034d94bcaab7cc632a1864 in term 0.
I20260923 20:51:37.527300 25074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7aae62b40f4320b22d65ccfb050ca4" candidate_uuid: "a1e9b62805034d94bcaab7cc632a1864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "526ecd5444064266a654322c8cb21962" is_pre_election: true
I20260923 20:51:37.527792 25074 raft_consensus.cc:2454] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e9b62805034d94bcaab7cc632a1864 in term 0.
I20260923 20:51:37.528143 25143 leader_election.cc:302] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32f777c8850048cc8aaffa69109648f5, a1e9b62805034d94bcaab7cc632a1864; no voters:
I20260923 20:51:37.528607 25278 raft_consensus.cc:2781] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:51:37.528776 25278 raft_consensus.cc:482] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:51:37.528921 25278 raft_consensus.cc:3037] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:37.531127 25278 raft_consensus.cc:504] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.531831 25278 leader_election.cc:288] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [CANDIDATE]: Term 1 election: Requested vote from peers 32f777c8850048cc8aaffa69109648f5 (127.5.71.129:38909), 526ecd5444064266a654322c8cb21962 (127.5.71.130:37499)
I20260923 20:51:37.532392 25074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7aae62b40f4320b22d65ccfb050ca4" candidate_uuid: "a1e9b62805034d94bcaab7cc632a1864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "526ecd5444064266a654322c8cb21962"
I20260923 20:51:37.532379 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e7aae62b40f4320b22d65ccfb050ca4" candidate_uuid: "a1e9b62805034d94bcaab7cc632a1864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32f777c8850048cc8aaffa69109648f5"
I20260923 20:51:37.532604 24940 raft_consensus.cc:3037] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:37.532604 25074 raft_consensus.cc:3037] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:37.535964 25074 raft_consensus.cc:2454] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e9b62805034d94bcaab7cc632a1864 in term 1.
I20260923 20:51:37.535995 24940 raft_consensus.cc:2454] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e9b62805034d94bcaab7cc632a1864 in term 1.
I20260923 20:51:37.536445 25143 leader_election.cc:302] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 526ecd5444064266a654322c8cb21962, a1e9b62805034d94bcaab7cc632a1864; no voters:
I20260923 20:51:37.536787 25278 raft_consensus.cc:2781] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:37.537246 25278 raft_consensus.cc:686] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 1 LEADER]: Becoming Leader. State: Replica: a1e9b62805034d94bcaab7cc632a1864, State: Running, Role: LEADER
I20260923 20:51:37.537761 25278 consensus_queue.cc:237] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } }
I20260923 20:51:37.542840 24795 catalog_manager.cc:5495] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1e9b62805034d94bcaab7cc632a1864 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "a1e9b62805034d94bcaab7cc632a1864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1e9b62805034d94bcaab7cc632a1864" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 } health_report { overall_health: UNKNOWN } } }
I20260923 20:51:37.603725 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.597581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.5.71.131:42907)
0923 20:51:37.597782 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.597787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.597805 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:51:37.597901 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.597905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.597989 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.598123 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:51:37.598633 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.599390 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.601142 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.601700 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.601811 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.602361 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:51:37.602968 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:51:37.603133 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.603260 (+ 127us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.603358 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:51:37.612938 25074 raft_consensus.cc:1261] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 1 FOLLOWER]: Refusing update from remote peer a1e9b62805034d94bcaab7cc632a1864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:37.612938 24940 raft_consensus.cc:1261] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 1 FOLLOWER]: Refusing update from remote peer a1e9b62805034d94bcaab7cc632a1864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:37.614144 25282 consensus_queue.cc:1035] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f777c8850048cc8aaffa69109648f5" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 38909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:37.614681 25278 consensus_queue.cc:1035] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "526ecd5444064266a654322c8cb21962" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 37499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:37.642079 24796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:37.642323 24796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:37.646027 24796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53f15cd71e3c48c3a3e4cbed3abfc18d: Sending DeleteTablet for 3 replicas of tablet 1e7aae62b40f4320b22d65ccfb050ca4
I20260923 20:51:37.647189 25188 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e7aae62b40f4320b22d65ccfb050ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:37 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:34458
I20260923 20:51:37.647372 25054 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e7aae62b40f4320b22d65ccfb050ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:37 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:51984
I20260923 20:51:37.648407 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24857
I20260923 20:51:37.648504 25295 tablet_replica.cc:330] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: stopping tablet replica
I20260923 20:51:37.649046 25295 raft_consensus.cc:2229] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:51:37.649053 24920 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e7aae62b40f4320b22d65ccfb050ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:37 UTC) from {username='kudu', principal='kudu/127.5.71.190@KRBTEST.COM'} at 127.0.0.1:59684
I20260923 20:51:37.649865 25295 raft_consensus.cc:2258] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:37.650795 25296 tablet_replica.cc:330] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: stopping tablet replica
I20260923 20:51:37.650985 25297 tablet_replica.cc:330] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: stopping tablet replica
I20260923 20:51:37.651285 25296 raft_consensus.cc:2229] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:37.651358 25297 raft_consensus.cc:2229] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:37.651836 25296 raft_consensus.cc:2258] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:37.651839 25297 raft_consensus.cc:2258] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:37.654228 25296 ts_tablet_manager.cc:1905] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:37.655596 25297 ts_tablet_manager.cc:1905] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:37.661643 25296 ts_tablet_manager.cc:1918] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:37.662045 25296 log.cc:1196] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/1e7aae62b40f4320b22d65ccfb050ca4
I20260923 20:51:37.662736 25296 ts_tablet_manager.cc:1939] T 1e7aae62b40f4320b22d65ccfb050ca4 P 526ecd5444064266a654322c8cb21962: Deleting consensus metadata
I20260923 20:51:37.662904 25297 ts_tablet_manager.cc:1918] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Sep 23 20:51:37 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.129@KRBTEST.COM
I20260923 20:51:37.664973 24782 catalog_manager.cc:4841] TS 526ecd5444064266a654322c8cb21962 (127.5.71.130:37499): tablet 1e7aae62b40f4320b22d65ccfb050ca4 (table test-table [id=7cebd832829141f49a7884dc5784d288]) successfully deleted
I20260923 20:51:37.665199 25295 ts_tablet_manager.cc:1905] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:37.671813 25297 log.cc:1196] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/1e7aae62b40f4320b22d65ccfb050ca4
I20260923 20:51:37.672910 25297 ts_tablet_manager.cc:1939] T 1e7aae62b40f4320b22d65ccfb050ca4 P 32f777c8850048cc8aaffa69109648f5: Deleting consensus metadata
I20260923 20:51:37.673676 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.651967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.5.71.129:38909)
0923 20:51:37.652162 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.652169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.652578 (+ 409us) server_negotiation.cc:407] Connection header received
0923 20:51:37.652902 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.652910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.652997 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.653076 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:37.655118 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.656262 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.658154 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.658765 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.658864 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.664753 (+ 5889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:37.664772 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:37.664791 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:37.664837 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:37.669792 (+ 4955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.670799 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.670810 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.670814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.670978 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.671509 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.671515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.671519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.672017 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:37.672346 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.673061 (+ 715us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.673298 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:51:37.675385 24980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:37.676841 25295 ts_tablet_manager.cc:1918] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:37.677085 25295 log.cc:1196] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/1e7aae62b40f4320b22d65ccfb050ca4
I20260923 20:51:37.677755 25295 ts_tablet_manager.cc:1939] T 1e7aae62b40f4320b22d65ccfb050ca4 P a1e9b62805034d94bcaab7cc632a1864: Deleting consensus metadata
I20260923 20:51:37.679688 24780 catalog_manager.cc:4841] TS a1e9b62805034d94bcaab7cc632a1864 (127.5.71.131:42907): tablet 1e7aae62b40f4320b22d65ccfb050ca4 (table test-table [id=7cebd832829141f49a7884dc5784d288]) successfully deleted
I20260923 20:51:37.833927 24782 catalog_manager.cc:4841] TS 32f777c8850048cc8aaffa69109648f5 (127.5.71.129:38909): tablet 1e7aae62b40f4320b22d65ccfb050ca4 (table test-table [id=7cebd832829141f49a7884dc5784d288]) successfully deleted
I20260923 20:51:37.834477 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24857
I20260923 20:51:37.855242 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24991
Sep 23 20:51:37 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.130@KRBTEST.COM
I20260923 20:51:37.873413 25271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.856143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.5.71.130:37499)
0923 20:51:37.856342 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.856348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.856493 (+ 145us) server_negotiation.cc:407] Connection header received
0923 20:51:37.856758 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.856763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.856858 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.856986 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:37.858320 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.859139 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.860585 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.861067 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.861237 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.866204 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:37.866223 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:37.866242 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:37.866288 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:37.869810 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.870629 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.870638 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.870640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.870801 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:37.871241 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:37.871248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:37.871251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:37.871718 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:37.872147 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:37.872539 (+ 392us) server_negotiation.cc:299] Negotiation successful
0923 20:51:37.872767 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:51:37.874152 25114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:37.967103 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24991
I20260923 20:51:37.985985 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25125
Sep 23 20:51:37 dist-test-slave-dn29 krb5kdc[24751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.131@KRBTEST.COM
I20260923 20:51:38.004444 25267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:37.986981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34480 (local address 127.5.71.131:42907)
0923 20:51:37.987201 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:37.987211 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:37.987235 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:51:37.987308 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:37.987315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:37.987389 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:37.987527 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:37.988993 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.989901 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.991367 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:37.991999 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:37.992171 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:37.997548 (+ 5377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:37.997567 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:37.997586 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:37.997640 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:38.001232 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:38.002025 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:38.002035 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:38.002037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:38.002229 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:38.002640 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:38.002647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:38.002654 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:38.003022 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:38.003343 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:38.003673 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 20:51:38.003940 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:51:38.005477 25248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:38.134068 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25125
I20260923 20:51:38.154361 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 24764
I20260923 20:51:38.155552 24826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:38.271242 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 24764
2026-09-23T20:51:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4808 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T20:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:38Z Disabled control of system clock
I20260923 20:51:38.357146 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43481
--webserver_interface=127.5.71.190
--webserver_port=0
--builtin_ntp_servers=127.5.71.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:43481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 20:51:38.724423 25306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:38.724740 25306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:38.724859 25306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:38.732191 25306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:38.732304 25306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:38.732383 25306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:38.732466 25306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:38.732527 25306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:38.742101 25306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:43481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:43481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:38.743506 25306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:38.745452 25306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:38.759248 25311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:38.759256 25314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:38.759167 25312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:38.760066 25306 server_base.cc:1034] running on GCE node
I20260923 20:51:38.760886 25306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:38.762333 25306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:38.763553 25306 hybrid_clock.cc:648] HybridClock initialized: now 1790196698763514 us; error 36 us; skew 500 ppm
I20260923 20:51:38.763974 25306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:38.766335 25306 webserver.cc:460] Webserver started at http://127.5.71.190:46381/ using document root <none> and password file <none>
I20260923 20:51:38.766992 25306 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:38.767078 25306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:38.767347 25306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:38.770319 25306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "25c79d62c2454974a339a83eec2677d8"
format_stamp: "Formatted at 2026-09-23 20:51:38 on dist-test-slave-dn29"
server_key: "964f773453a13bc30a6dbdbdaa6a76e0"
server_key_iv: "2d833a06176d94d5a675febdefa85c38"
server_key_version: "encryptionkey@0"
I20260923 20:51:38.771075 25306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "25c79d62c2454974a339a83eec2677d8"
format_stamp: "Formatted at 2026-09-23 20:51:38 on dist-test-slave-dn29"
server_key: "964f773453a13bc30a6dbdbdaa6a76e0"
server_key_iv: "2d833a06176d94d5a675febdefa85c38"
server_key_version: "encryptionkey@0"
I20260923 20:51:38.776333 25306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260923 20:51:38.779985 25320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:38.781541 25306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:38.781719 25306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "25c79d62c2454974a339a83eec2677d8"
format_stamp: "Formatted at 2026-09-23 20:51:38 on dist-test-slave-dn29"
server_key: "964f773453a13bc30a6dbdbdaa6a76e0"
server_key_iv: "2d833a06176d94d5a675febdefa85c38"
server_key_version: "encryptionkey@0"
I20260923 20:51:38.781932 25306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:38.810877 25306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:38.811827 25306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:38.812099 25306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:38.833068 25306 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:43481
I20260923 20:51:38.833077 25371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:43481 every 8 connection(s)
I20260923 20:51:38.834596 25306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:38.836127 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25306
I20260923 20:51:38.836318 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:38.836889 5406 external_mini_cluster.cc:1428] Setting key bc655d1e798b11e92047979780405cca
I20260923 20:51:38.840124 25372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:38.855334 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:38.839141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.5.71.190:43481)
0923 20:51:38.840383 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:38.840400 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:38.840459 (+ 59us) server_negotiation.cc:407] Connection header received
0923 20:51:38.843152 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:38.843183 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:38.843909 (+ 726us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:38.844393 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:38.845141 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:38.847467 (+ 2326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:38.849093 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:38.850257 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:38.850464 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:38.851983 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:38.851994 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:38.852019 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:38.852895 (+ 876us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:38.853004 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:38.853037 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:38.853189 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:38.853444 (+ 255us) server_negotiation.cc:299] Negotiation successful
0923 20:51:38.853876 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":381,"threads_started":1}
I20260923 20:51:38.856806 25372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8: Bootstrap starting.
I20260923 20:51:38.860258 25372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:38.861352 25372 log.cc:824] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:38.864424 25372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8: No bootstrap required, opened a new log
I20260923 20:51:38.872642 25372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c79d62c2454974a339a83eec2677d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43481 } }
I20260923 20:51:38.873118 25372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:38.873252 25372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c79d62c2454974a339a83eec2677d8, State: Initialized, Role: FOLLOWER
I20260923 20:51:38.873965 25372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c79d62c2454974a339a83eec2677d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43481 } }
I20260923 20:51:38.874249 25372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:38.874413 25372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:38.874606 25372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:38.877266 25372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c79d62c2454974a339a83eec2677d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43481 } }
I20260923 20:51:38.877846 25372 leader_election.cc:302] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25c79d62c2454974a339a83eec2677d8; no voters:
I20260923 20:51:38.878556 25372 leader_election.cc:288] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:38.878743 25377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:38.879518 25377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [term 1 LEADER]: Becoming Leader. State: Replica: 25c79d62c2454974a339a83eec2677d8, State: Running, Role: LEADER
I20260923 20:51:38.880188 25377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c79d62c2454974a339a83eec2677d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43481 } }
I20260923 20:51:38.880939 25372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:38.883001 25378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25c79d62c2454974a339a83eec2677d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c79d62c2454974a339a83eec2677d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43481 } } }
I20260923 20:51:38.883001 25379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25c79d62c2454974a339a83eec2677d8. Latest consensus state: current_term: 1 leader_uuid: "25c79d62c2454974a339a83eec2677d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c79d62c2454974a339a83eec2677d8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 43481 } } }
I20260923 20:51:38.883562 25379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:38.883563 25378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:38.886564 25382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:38.893846 25382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:38.904726 25382 catalog_manager.cc:1347] Generated new cluster ID: 78a396d2d0ef4f98aebd075252eed303
I20260923 20:51:38.904922 25382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:38.925457 25382 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:38.926960 25382 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:38.937934 25382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8: Generated new TSK 0
I20260923 20:51:38.938896 25382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:38.947289 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.129:0
--local_ip_for_outbound_sockets=127.5.71.129
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43481
--builtin_ntp_servers=127.5.71.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 20:51:39.307746 25396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:39.308158 25396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:39.308344 25396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:39.316119 25396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:39.316274 25396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:39.316510 25396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.129
I20260923 20:51:39.325876 25396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.71.129
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.129
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:39.327271 25396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:39.329205 25396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:39.343458 25404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:39.343918 25401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:39.344043 25402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:39.344933 25396 server_base.cc:1034] running on GCE node
I20260923 20:51:39.345745 25396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:39.346765 25396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:39.348026 25396 hybrid_clock.cc:648] HybridClock initialized: now 1790196699347957 us; error 55 us; skew 500 ppm
I20260923 20:51:39.348443 25396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:39.351920 25396 webserver.cc:460] Webserver started at http://127.5.71.129:37189/ using document root <none> and password file <none>
I20260923 20:51:39.352661 25396 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:39.352798 25396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:39.353138 25396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:39.356263 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data/instance:
uuid: "6904343a3a904391a82cf6c6cd558229"
format_stamp: "Formatted at 2026-09-23 20:51:39 on dist-test-slave-dn29"
server_key: "3f42aac3b1c3c9f36c7049de0b44eace"
server_key_iv: "1dfa0b5ba31c705b4eb7f4da89c42e43"
server_key_version: "encryptionkey@0"
I20260923 20:51:39.357131 25396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance:
uuid: "6904343a3a904391a82cf6c6cd558229"
format_stamp: "Formatted at 2026-09-23 20:51:39 on dist-test-slave-dn29"
server_key: "3f42aac3b1c3c9f36c7049de0b44eace"
server_key_iv: "1dfa0b5ba31c705b4eb7f4da89c42e43"
server_key_version: "encryptionkey@0"
I20260923 20:51:39.362391 25396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:51:39.366220 25410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:39.367853 25396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260923 20:51:39.368038 25396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal
uuid: "6904343a3a904391a82cf6c6cd558229"
format_stamp: "Formatted at 2026-09-23 20:51:39 on dist-test-slave-dn29"
server_key: "3f42aac3b1c3c9f36c7049de0b44eace"
server_key_iv: "1dfa0b5ba31c705b4eb7f4da89c42e43"
server_key_version: "encryptionkey@0"
I20260923 20:51:39.368283 25396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:39.388931 25396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:39.389894 25396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:39.390177 25396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:39.391165 25396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:39.392848 25396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:39.392940 25396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:39.393016 25396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:39.393070 25396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:39.429273 25396 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.129:33639
I20260923 20:51:39.429373 25522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.129:33639 every 8 connection(s)
I20260923 20:51:39.430656 25396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/data/info.pb
I20260923 20:51:39.435431 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25396
I20260923 20:51:39.435606 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal/instance
I20260923 20:51:39.436131 5406 external_mini_cluster.cc:1428] Setting key 156880e99be9e3d9465a63f4216ec0e4
I20260923 20:51:39.439074 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.130:0
--local_ip_for_outbound_sockets=127.5.71.130
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43481
--builtin_ntp_servers=127.5.71.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 20:51:39.445495 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:39.433521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.129:53927 (local address 127.5.71.190:43481)
0923 20:51:39.434048 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:39.434058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:39.435187 (+ 1129us) server_negotiation.cc:407] Connection header received
0923 20:51:39.437851 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:39.437855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:39.437976 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:39.438095 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:39.439564 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.440409 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.442192 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.442682 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.442838 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:39.444492 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:39.444497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:39.444501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:39.444523 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:39.444586 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:39.444600 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:39.444716 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:39.444991 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 20:51:39.445107 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":290,"threads_started":1}
I20260923 20:51:39.447126 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:39.434036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43481 (local address 127.5.71.129:53927)
0923 20:51:39.434968 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:39.435013 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:39.437524 (+ 2511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:39.438518 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:39.438527 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:39.438632 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:39.439380 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:39.439402 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.440594 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.440599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:39.442017 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:39.442030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.442942 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.442950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:39.443224 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:39.444089 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:39.444092 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:39.444140 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:39.444246 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:39.444716 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:39.444732 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:39.444745 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 20:51:39.444978 (+ 233us) client_negotiation.cc:240] Negotiation successful
0923 20:51:39.445469 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":281,"threads_started":1}
I20260923 20:51:39.448799 25523 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43481
I20260923 20:51:39.449162 25523 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:39.450193 25523 heartbeater.cc:511] Master 127.5.71.190:43481 requested a full tablet report, sending...
I20260923 20:51:39.452576 25337 ts_manager.cc:194] Registered new tserver with Master: 6904343a3a904391a82cf6c6cd558229 (127.5.71.129:33639)
I20260923 20:51:39.454046 25337 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.129:53927
W20260923 20:51:39.782832 25527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:39.783175 25527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:39.783344 25527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:39.791172 25527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:39.791293 25527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:39.791486 25527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.130
I20260923 20:51:39.800861 25527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.71.130
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.130
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:39.802225 25527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:39.804168 25527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:39.818213 25535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:39.818292 25532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:39.818246 25533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:39.818707 25527 server_base.cc:1034] running on GCE node
I20260923 20:51:39.819621 25527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:39.820684 25527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:39.821920 25527 hybrid_clock.cc:648] HybridClock initialized: now 1790196699821872 us; error 27 us; skew 500 ppm
I20260923 20:51:39.822304 25527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:39.825019 25527 webserver.cc:460] Webserver started at http://127.5.71.130:46347/ using document root <none> and password file <none>
I20260923 20:51:39.825763 25527 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:39.825886 25527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:39.826267 25527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:39.829373 25527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data/instance:
uuid: "45a9886d92974687b74abf7e5150510c"
format_stamp: "Formatted at 2026-09-23 20:51:39 on dist-test-slave-dn29"
server_key: "efb8caed1766a9c36b026d925eaf70cc"
server_key_iv: "1e6b653907e35b5bee8e1a76343149fc"
server_key_version: "encryptionkey@0"
I20260923 20:51:39.830202 25527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance:
uuid: "45a9886d92974687b74abf7e5150510c"
format_stamp: "Formatted at 2026-09-23 20:51:39 on dist-test-slave-dn29"
server_key: "efb8caed1766a9c36b026d925eaf70cc"
server_key_iv: "1e6b653907e35b5bee8e1a76343149fc"
server_key_version: "encryptionkey@0"
I20260923 20:51:39.836095 25527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:51:39.839901 25541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:39.841369 25527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:39.841566 25527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal
uuid: "45a9886d92974687b74abf7e5150510c"
format_stamp: "Formatted at 2026-09-23 20:51:39 on dist-test-slave-dn29"
server_key: "efb8caed1766a9c36b026d925eaf70cc"
server_key_iv: "1e6b653907e35b5bee8e1a76343149fc"
server_key_version: "encryptionkey@0"
I20260923 20:51:39.841804 25527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:39.868112 25527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:39.869048 25527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:39.869385 25527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:39.870476 25527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:39.872268 25527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:39.872394 25527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:39.872546 25527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:39.872632 25527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:39.910014 25527 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.130:38171
I20260923 20:51:39.910116 25653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.130:38171 every 8 connection(s)
I20260923 20:51:39.911394 25527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/data/info.pb
I20260923 20:51:39.919382 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25527
I20260923 20:51:39.919538 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal/instance
I20260923 20:51:39.920095 5406 external_mini_cluster.cc:1428] Setting key c592e0c73d4c83e9412847b874855ae6
I20260923 20:51:39.923306 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.71.131:0
--local_ip_for_outbound_sockets=127.5.71.131
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43481
--builtin_ntp_servers=127.5.71.148:41871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 20:51:39.926450 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:39.914323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.130:60891 (local address 127.5.71.190:43481)
0923 20:51:39.914546 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:39.914553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:39.916132 (+ 1579us) server_negotiation.cc:407] Connection header received
0923 20:51:39.917996 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:39.918002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:39.918126 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:39.918247 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:39.919773 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.920699 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.922861 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.923324 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.923490 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:39.925410 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:39.925416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:39.925420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:39.925441 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:39.925474 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:39.925488 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:39.925593 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:39.925883 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 20:51:39.926012 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 20:51:39.927603 25656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:39.914803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43481 (local address 127.5.71.130:60891)
0923 20:51:39.915952 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:39.915992 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:39.917689 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:39.918518 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:39.918528 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:39.918697 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:39.919427 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:39.919447 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.920888 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.920894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:39.922678 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:39.922692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:39.923599 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:39.923620 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:39.923971 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:39.925000 (+ 1029us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:39.925005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:39.925054 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:39.925158 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:39.925617 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:39.925630 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:39.925640 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 20:51:39.925867 (+ 227us) client_negotiation.cc:240] Negotiation successful
0923 20:51:39.926381 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":293,"threads_started":1}
I20260923 20:51:39.929391 25654 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43481
I20260923 20:51:39.929738 25654 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:39.930576 25654 heartbeater.cc:511] Master 127.5.71.190:43481 requested a full tablet report, sending...
I20260923 20:51:39.932430 25337 ts_manager.cc:194] Registered new tserver with Master: 45a9886d92974687b74abf7e5150510c (127.5.71.130:38171)
I20260923 20:51:39.933733 25337 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.130:60891
W20260923 20:51:40.279320 25657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:40.279663 25657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:40.279762 25657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:40.288620 25657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:40.288751 25657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:40.288921 25657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.71.131
I20260923 20:51:40.299881 25657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:41871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.71.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.71.131
--webserver_port=0
--tserver_master_addrs=127.5.71.190:43481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.71.131
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:40.301364 25657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:40.303316 25657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:40.317420 25663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:40.317396 25662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:40.317353 25665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:40.318578 25657 server_base.cc:1034] running on GCE node
I20260923 20:51:40.319262 25657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:40.320369 25657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:40.321652 25657 hybrid_clock.cc:648] HybridClock initialized: now 1790196700321530 us; error 32 us; skew 500 ppm
I20260923 20:51:40.322026 25657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260923 20:51:40.324563 25657 webserver.cc:460] Webserver started at http://127.5.71.131:45627/ using document root <none> and password file <none>
I20260923 20:51:40.325260 25657 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:40.325371 25657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:40.325707 25657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:40.328780 25657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data/instance:
uuid: "8a845c76a00649b59975a784ccc8eeb7"
format_stamp: "Formatted at 2026-09-23 20:51:40 on dist-test-slave-dn29"
server_key: "ad72eba08fab5bb89fba1478124c1c8e"
server_key_iv: "cf3a703993e66a919b86e2a7b426f1fb"
server_key_version: "encryptionkey@0"
I20260923 20:51:40.329613 25657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance:
uuid: "8a845c76a00649b59975a784ccc8eeb7"
format_stamp: "Formatted at 2026-09-23 20:51:40 on dist-test-slave-dn29"
server_key: "ad72eba08fab5bb89fba1478124c1c8e"
server_key_iv: "cf3a703993e66a919b86e2a7b426f1fb"
server_key_version: "encryptionkey@0"
I20260923 20:51:40.334965 25657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:51:40.338866 25671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:40.340618 25657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:40.340760 25657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal
uuid: "8a845c76a00649b59975a784ccc8eeb7"
format_stamp: "Formatted at 2026-09-23 20:51:40 on dist-test-slave-dn29"
server_key: "ad72eba08fab5bb89fba1478124c1c8e"
server_key_iv: "cf3a703993e66a919b86e2a7b426f1fb"
server_key_version: "encryptionkey@0"
I20260923 20:51:40.340943 25657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:40.373588 25657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:40.374792 25657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:40.375200 25657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:40.376538 25657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 20:51:40.378070 25657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 20:51:40.378178 25657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:40.378261 25657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 20:51:40.378337 25657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:40.419462 25657 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.131:34753
I20260923 20:51:40.419517 25783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.131:34753 every 8 connection(s)
I20260923 20:51:40.420933 25657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/data/info.pb
I20260923 20:51:40.423482 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25657
I20260923 20:51:40.423696 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal/instance
I20260923 20:51:40.424250 5406 external_mini_cluster.cc:1428] Setting key 8758c18aa5817192b5903e52386636a4
I20260923 20:51:40.437384 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.424491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:49063 (local address 127.5.71.190:43481)
0923 20:51:40.424738 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.424746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.426741 (+ 1995us) server_negotiation.cc:407] Connection header received
0923 20:51:40.428999 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.429004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.429080 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.429170 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:40.430685 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.431541 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.434171 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.434724 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.434836 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.436573 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:40.436578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:40.436581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:40.436609 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:40.436645 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:40.436657 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:40.436737 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.437041 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.437132 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 20:51:40.438764 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.425092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:43481 (local address 127.5.71.131:49063)
0923 20:51:40.426421 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.426484 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:40.428644 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:40.429451 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:40.429466 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:40.429601 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:51:40.430470 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.430498 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.431847 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.431853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.433969 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.433987 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.434990 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.434998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.435343 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.436206 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 20:51:40.436208 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:51:40.436250 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 20:51:40.436342 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:51:40.436767 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:51:40.436777 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:51:40.436784 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 20:51:40.436950 (+ 166us) client_negotiation.cc:240] Negotiation successful
0923 20:51:40.437304 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":413,"threads_started":1}
I20260923 20:51:40.440383 25784 heartbeater.cc:347] Connected to a master server at 127.5.71.190:43481
I20260923 20:51:40.440734 25784 heartbeater.cc:464] Registering TS with master...
I20260923 20:51:40.441486 25784 heartbeater.cc:511] Master 127.5.71.190:43481 requested a full tablet report, sending...
I20260923 20:51:40.442939 25337 ts_manager.cc:194] Registered new tserver with Master: 8a845c76a00649b59975a784ccc8eeb7 (127.5.71.131:34753)
I20260923 20:51:40.443893 25337 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.71.131:49063
I20260923 20:51:40.453477 5406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 20:51:40.458199 25523 heartbeater.cc:503] Master 127.5.71.190:43481 was elected leader, sending a full tablet report...
I20260923 20:51:40.464872 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.458213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.5.71.190:43481)
0923 20:51:40.458487 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.458495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.459012 (+ 517us) server_negotiation.cc:407] Connection header received
0923 20:51:40.459159 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.459165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.459279 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.459371 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:40.460043 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.461041 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.462392 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.462819 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.462936 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.464196 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:40.464201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:40.464204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:40.464224 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:40.464257 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:40.464269 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:40.464346 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.464510 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.464594 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 20:51:40.469427 25337 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 20:51:40.477535 25337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 20:51:40.506682 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.494202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42368)
0923 20:51:40.496659 (+ 2457us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.496685 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:40.496815 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:40.497597 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:40.497603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:40.497644 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:40.498135 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.498146 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.500547 (+ 2401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.500553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.502869 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.502883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.505514 (+ 2631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.505521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.505777 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.505846 (+ 69us) client_negotiation.cc:769] Sending connection context
0923 20:51:40.505973 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 20:51:40.506185 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2284,"mutex_wait_us":1,"thread_start_us":2082,"threads_started":1}
I20260923 20:51:40.507447 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.492608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.131:34753 (local address 127.0.0.1:53630)
0923 20:51:40.493509 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.493572 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:40.493887 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:40.495144 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:40.495156 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:40.495197 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:40.495654 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.495669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.498138 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.498143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.503082 (+ 4939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.503089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.506530 (+ 3441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.506538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.506782 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.506828 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 20:51:40.506905 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 20:51:40.507034 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":572,"thread_start_us":370,"threads_started":1}
I20260923 20:51:40.508788 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.493012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.5.71.131:34753)
0923 20:51:40.493855 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.493866 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.493906 (+ 40us) server_negotiation.cc:407] Connection header received
0923 20:51:40.494215 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.494225 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.494555 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.494885 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:40.495954 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.497934 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.503293 (+ 5359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.506281 (+ 2988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.507713 (+ 1432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.508085 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.508197 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.508389 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":467,"threads_started":1}
I20260923 20:51:40.509145 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.494273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.5.71.129:33639)
0923 20:51:40.494914 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.494923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.496969 (+ 2046us) server_negotiation.cc:407] Connection header received
0923 20:51:40.497082 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.497091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.497370 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.498404 (+ 1034us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:40.498694 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.500385 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.503391 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.505267 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.508203 (+ 2936us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.508516 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.508593 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.508808 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":332,"threads_started":1}
I20260923 20:51:40.513588 25458 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2988da861e4221b14c18819df2eae5 (DEFAULT_TABLE table=test-table [id=427e5e9f140a40109e32333a12900b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:40.515333 25458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2988da861e4221b14c18819df2eae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:40.517925 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.496391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38171 (local address 127.0.0.1:48146)
0923 20:51:40.507841 (+ 11450us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.507866 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:40.507987 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:40.510175 (+ 2188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:40.510180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:40.510217 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:40.510413 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.510426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.512841 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.512848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.514826 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.514859 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.516979 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.516986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.517242 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.517289 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 20:51:40.517383 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 20:51:40.517552 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11090,"thread_start_us":11631,"threads_started":1}
I20260923 20:51:40.519205 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.496944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.5.71.130:38171)
0923 20:51:40.508813 (+ 11869us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.508831 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.508904 (+ 73us) server_negotiation.cc:407] Connection header received
0923 20:51:40.509028 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.509039 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.509351 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.509581 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:40.510737 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.512643 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.515124 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.516757 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.518217 (+ 1460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.518527 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.518616 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.518812 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11700,"thread_start_us":11584,"threads_started":1}
I20260923 20:51:40.522045 25719 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2988da861e4221b14c18819df2eae5 (DEFAULT_TABLE table=test-table [id=427e5e9f140a40109e32333a12900b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:40.523990 25719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2988da861e4221b14c18819df2eae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:40.524367 25589 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2988da861e4221b14c18819df2eae5 (DEFAULT_TABLE table=test-table [id=427e5e9f140a40109e32333a12900b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 20:51:40.525758 25589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2988da861e4221b14c18819df2eae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:40.537451 25802 tablet_bootstrap.cc:492] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Bootstrap starting.
I20260923 20:51:40.541635 25802 tablet_bootstrap.cc:654] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:40.541850 25803 tablet_bootstrap.cc:492] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Bootstrap starting.
I20260923 20:51:40.543304 25802 log.cc:824] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:40.543401 25804 tablet_bootstrap.cc:492] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Bootstrap starting.
I20260923 20:51:40.546517 25803 tablet_bootstrap.cc:654] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:40.547624 25803 log.cc:824] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:40.548348 25804 tablet_bootstrap.cc:654] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:40.548894 25802 tablet_bootstrap.cc:492] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: No bootstrap required, opened a new log
I20260923 20:51:40.549206 25802 ts_tablet_manager.cc:1397] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260923 20:51:40.549762 25804 log.cc:824] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:40.551294 25803 tablet_bootstrap.cc:492] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: No bootstrap required, opened a new log
I20260923 20:51:40.551612 25803 ts_tablet_manager.cc:1397] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260923 20:51:40.553687 25804 tablet_bootstrap.cc:492] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: No bootstrap required, opened a new log
I20260923 20:51:40.554008 25804 ts_tablet_manager.cc:1397] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260923 20:51:40.560770 25803 raft_consensus.cc:348] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.561214 25803 raft_consensus.cc:374] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:40.561314 25803 raft_consensus.cc:729] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a9886d92974687b74abf7e5150510c, State: Initialized, Role: FOLLOWER
I20260923 20:51:40.561178 25802 raft_consensus.cc:348] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.561654 25802 raft_consensus.cc:374] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:40.561756 25802 raft_consensus.cc:729] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a845c76a00649b59975a784ccc8eeb7, State: Initialized, Role: FOLLOWER
I20260923 20:51:40.562237 25803 consensus_queue.cc:260] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.562376 25802 consensus_queue.cc:260] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.563496 25654 heartbeater.cc:503] Master 127.5.71.190:43481 was elected leader, sending a full tablet report...
I20260923 20:51:40.564337 25803 ts_tablet_manager.cc:1428] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260923 20:51:40.565338 25784 heartbeater.cc:503] Master 127.5.71.190:43481 was elected leader, sending a full tablet report...
I20260923 20:51:40.566246 25802 ts_tablet_manager.cc:1428] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260923 20:51:40.566596 25804 raft_consensus.cc:348] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.567099 25804 raft_consensus.cc:374] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:40.567229 25804 raft_consensus.cc:729] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6904343a3a904391a82cf6c6cd558229, State: Initialized, Role: FOLLOWER
I20260923 20:51:40.568033 25804 consensus_queue.cc:260] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.569869 25804 ts_tablet_manager.cc:1428] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260923 20:51:40.641965 25809 raft_consensus.cc:482] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 20:51:40.642351 25809 raft_consensus.cc:504] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.644413 25809 leader_election.cc:288] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6904343a3a904391a82cf6c6cd558229 (127.5.71.129:33639), 45a9886d92974687b74abf7e5150510c (127.5.71.130:38171)
I20260923 20:51:40.651147 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.644683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.5.71.131:46711)
0923 20:51:40.644898 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.644920 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:40.645059 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:40.645551 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:40.645556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:40.645609 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:40.645794 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.645802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.647147 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.647153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.649206 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.649225 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.650523 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.650527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.650660 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.650683 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 20:51:40.650816 (+ 133us) client_negotiation.cc:240] Negotiation successful
0923 20:51:40.650920 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":1}
I20260923 20:51:40.651247 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.644852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:46711 (local address 127.5.71.129:33639)
0923 20:51:40.645064 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.645070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.645095 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:51:40.645177 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.645183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.645344 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.645446 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:40.645981 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.647025 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.649360 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.650382 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.650557 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.650632 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.650863 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.650998 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 20:51:40.652275 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.644921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.71.131:55503 (local address 127.5.71.130:38171)
0923 20:51:40.645135 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.645143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.645932 (+ 789us) server_negotiation.cc:407] Connection header received
0923 20:51:40.646156 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.646161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.646235 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.646335 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 20:51:40.646932 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.648043 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.650309 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.651436 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.651599 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.651667 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.651971 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.652067 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 20:51:40.652384 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.644701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.130:38171 (local address 127.5.71.131:55503)
0923 20:51:40.645818 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.645835 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:51:40.645991 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:51:40.646387 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:51:40.646391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:51:40.646417 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 20:51:40.646780 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.646791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.648210 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.648213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.650124 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:51:40.650136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.651582 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.651588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:51:40.651749 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.651827 (+ 78us) client_negotiation.cc:769] Sending connection context
0923 20:51:40.651917 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 20:51:40.652051 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":358,"thread_start_us":338,"threads_started":1}
I20260923 20:51:40.652365 25478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2988da861e4221b14c18819df2eae5" candidate_uuid: "8a845c76a00649b59975a784ccc8eeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6904343a3a904391a82cf6c6cd558229" is_pre_election: true
I20260923 20:51:40.653010 25478 raft_consensus.cc:2454] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a845c76a00649b59975a784ccc8eeb7 in term 0.
I20260923 20:51:40.653329 25609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2988da861e4221b14c18819df2eae5" candidate_uuid: "8a845c76a00649b59975a784ccc8eeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45a9886d92974687b74abf7e5150510c" is_pre_election: true
I20260923 20:51:40.653808 25609 raft_consensus.cc:2454] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a845c76a00649b59975a784ccc8eeb7 in term 0.
I20260923 20:51:40.653837 25672 leader_election.cc:302] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6904343a3a904391a82cf6c6cd558229, 8a845c76a00649b59975a784ccc8eeb7; no voters:
I20260923 20:51:40.654230 25809 raft_consensus.cc:2781] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 20:51:40.654384 25809 raft_consensus.cc:482] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 20:51:40.654498 25809 raft_consensus.cc:3037] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:40.656764 25809 raft_consensus.cc:504] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.657421 25809 leader_election.cc:288] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [CANDIDATE]: Term 1 election: Requested vote from peers 6904343a3a904391a82cf6c6cd558229 (127.5.71.129:33639), 45a9886d92974687b74abf7e5150510c (127.5.71.130:38171)
I20260923 20:51:40.658020 25609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2988da861e4221b14c18819df2eae5" candidate_uuid: "8a845c76a00649b59975a784ccc8eeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45a9886d92974687b74abf7e5150510c"
I20260923 20:51:40.658020 25478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2988da861e4221b14c18819df2eae5" candidate_uuid: "8a845c76a00649b59975a784ccc8eeb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6904343a3a904391a82cf6c6cd558229"
I20260923 20:51:40.658246 25609 raft_consensus.cc:3037] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:40.658257 25478 raft_consensus.cc:3037] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:40.661655 25478 raft_consensus.cc:2454] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a845c76a00649b59975a784ccc8eeb7 in term 1.
I20260923 20:51:40.661657 25609 raft_consensus.cc:2454] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a845c76a00649b59975a784ccc8eeb7 in term 1.
I20260923 20:51:40.662179 25675 leader_election.cc:302] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 45a9886d92974687b74abf7e5150510c, 6904343a3a904391a82cf6c6cd558229, 8a845c76a00649b59975a784ccc8eeb7; no voters:
I20260923 20:51:40.662608 25809 raft_consensus.cc:2781] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:40.663130 25809 raft_consensus.cc:686] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 1 LEADER]: Becoming Leader. State: Replica: 8a845c76a00649b59975a784ccc8eeb7, State: Running, Role: LEADER
W20260923 20:51:40.663595 25655 tablet.cc:2369] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:40.663686 25809 consensus_queue.cc:237] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } }
I20260923 20:51:40.669139 25336 catalog_manager.cc:5495] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a845c76a00649b59975a784ccc8eeb7 (127.5.71.131). New cstate: current_term: 1 leader_uuid: "8a845c76a00649b59975a784ccc8eeb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a845c76a00649b59975a784ccc8eeb7" member_type: VOTER last_known_addr { host: "127.5.71.131" port: 34753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 } health_report { overall_health: UNKNOWN } } }
W20260923 20:51:40.672750 25785 tablet.cc:2369] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 20:51:40.683470 25524 tablet.cc:2369] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:51:40.701102 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.695252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.5.71.131:34753)
0923 20:51:40.695447 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.695452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.695472 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:51:40.695662 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.695666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.695751 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.695975 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:51:40.696457 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.697293 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.698715 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.699211 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.699321 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.699980 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:51:40.700559 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:51:40.700691 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.700821 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.700894 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:51:40.709986 25478 raft_consensus.cc:1261] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 1 FOLLOWER]: Refusing update from remote peer 8a845c76a00649b59975a784ccc8eeb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:40.709986 25609 raft_consensus.cc:1261] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 1 FOLLOWER]: Refusing update from remote peer 8a845c76a00649b59975a784ccc8eeb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:51:40.711148 25812 consensus_queue.cc:1035] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6904343a3a904391a82cf6c6cd558229" member_type: VOTER last_known_addr { host: "127.5.71.129" port: 33639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:40.711586 25809 consensus_queue.cc:1035] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45a9886d92974687b74abf7e5150510c" member_type: VOTER last_known_addr { host: "127.5.71.130" port: 38171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:51:40.723898 25815 mvcc.cc:205] Tried to move back new op lower bound from 7332645686095798272 to 7332645685923340288. Current Snapshot: MvccSnapshot[applied={T|T < 7332645686095798272}]
I20260923 20:51:40.737522 25337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:40.737916 25337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 20:51:40.741645 25337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25c79d62c2454974a339a83eec2677d8: Sending DeleteTablet for 3 replicas of tablet 3b2988da861e4221b14c18819df2eae5
I20260923 20:51:40.742888 25719 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2988da861e4221b14c18819df2eae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:40 UTC) from {username='slave'} at 127.0.0.1:53630
I20260923 20:51:40.742960 25458 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2988da861e4221b14c18819df2eae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:40 UTC) from {username='slave'} at 127.0.0.1:42368
I20260923 20:51:40.744092 25589 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2988da861e4221b14c18819df2eae5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 20:51:40 UTC) from {username='slave'} at 127.0.0.1:48146
I20260923 20:51:40.744382 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25396
I20260923 20:51:40.754345 25825 tablet_replica.cc:330] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: stopping tablet replica
I20260923 20:51:40.755023 25825 raft_consensus.cc:2229] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:40.755759 25825 raft_consensus.cc:2258] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:40.758788 25824 tablet_replica.cc:330] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: stopping tablet replica
I20260923 20:51:40.759308 25824 raft_consensus.cc:2229] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 1 LEADER]: Raft consensus shutting down.
I20260923 20:51:40.760118 25824 raft_consensus.cc:2258] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:40.760496 25825 ts_tablet_manager.cc:1905] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:40.762537 25824 ts_tablet_manager.cc:1905] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:40.766928 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.745407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.5.71.129:33639)
0923 20:51:40.745631 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.745637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.758241 (+ 12604us) server_negotiation.cc:407] Connection header received
0923 20:51:40.758812 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.758821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.758926 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.759006 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:40.759682 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.760804 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.762707 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.763397 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.763497 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.765045 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:40.765057 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:40.765077 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:40.765957 (+ 880us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:40.766018 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:40.766046 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:40.766134 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.766368 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.766498 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260923 20:51:40.768162 25518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:40.768548 25826 tablet_replica.cc:330] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: stopping tablet replica
I20260923 20:51:40.769057 25826 raft_consensus.cc:2229] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 20:51:40.769673 25826 raft_consensus.cc:2258] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 20:51:40.771634 25824 ts_tablet_manager.cc:1918] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:40.772519 25824 log.cc:1196] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-2/wal/wals/3b2988da861e4221b14c18819df2eae5
I20260923 20:51:40.773257 25824 ts_tablet_manager.cc:1939] T 3b2988da861e4221b14c18819df2eae5 P 8a845c76a00649b59975a784ccc8eeb7: Deleting consensus metadata
I20260923 20:51:40.773784 25826 ts_tablet_manager.cc:1905] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 20:51:40.775638 25324 catalog_manager.cc:4841] TS 8a845c76a00649b59975a784ccc8eeb7 (127.5.71.131:34753): tablet 3b2988da861e4221b14c18819df2eae5 (table test-table [id=427e5e9f140a40109e32333a12900b17]) successfully deleted
I20260923 20:51:40.781524 25826 ts_tablet_manager.cc:1918] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:40.781714 25826 log.cc:1196] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-1/wal/wals/3b2988da861e4221b14c18819df2eae5
I20260923 20:51:40.782276 25826 ts_tablet_manager.cc:1939] T 3b2988da861e4221b14c18819df2eae5 P 45a9886d92974687b74abf7e5150510c: Deleting consensus metadata
I20260923 20:51:40.784307 25324 catalog_manager.cc:4841] TS 45a9886d92974687b74abf7e5150510c (127.5.71.130:38171): tablet 3b2988da861e4221b14c18819df2eae5 (table test-table [id=427e5e9f140a40109e32333a12900b17]) successfully deleted
I20260923 20:51:40.868860 25825 ts_tablet_manager.cc:1918] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 20:51:40.869133 25825 log.cc:1196] T 3b2988da861e4221b14c18819df2eae5 P 6904343a3a904391a82cf6c6cd558229: Deleting WAL directory at /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790196359367181-5406-0/minicluster-data/ts-0/wal/wals/3b2988da861e4221b14c18819df2eae5
I20260923 20:51:40.869577 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25396
W20260923 20:51:40.888590 25321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.71.129:33639 (error 108)
I20260923 20:51:40.888836 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25527
I20260923 20:51:40.890676 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.890113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42394)
0923 20:51:40.890306 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.890429 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260923 20:51:40.891057 25321 catalog_manager.cc:4584] TS 6904343a3a904391a82cf6c6cd558229 (127.5.71.129:33639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b2988da861e4221b14c18819df2eae5: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
I20260923 20:51:40.898113 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.889783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.5.71.130:38171)
0923 20:51:40.889983 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:40.889991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:40.890562 (+ 571us) server_negotiation.cc:407] Connection header received
0923 20:51:40.890751 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:40.890757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:40.890876 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:40.891014 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:40.891610 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.892822 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.894335 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:40.894977 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:40.895143 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:40.896327 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:40.896339 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:40.896362 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:40.897174 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:40.897259 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:40.897310 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:40.897436 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:40.897593 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 20:51:40.897711 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:51:40.898984 25649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:40.944928 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.944361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42398)
0923 20:51:40.944618 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.944749 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260923 20:51:40.946136 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:40.945691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42412)
0923 20:51:40.945875 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:40.945962 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 20:51:40.990320 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25527
I20260923 20:51:41.007066 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:41.006524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42426)
0923 20:51:41.006828 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:41.006917 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260923 20:51:41.008329 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:41.007952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42432)
0923 20:51:41.008098 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:41.008189 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 20:51:41.014528 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25657
I20260923 20:51:41.022921 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:41.015377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53656 (local address 127.5.71.131:34753)
0923 20:51:41.015587 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:41.015591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:41.015610 (+ 19us) server_negotiation.cc:407] Connection header received
0923 20:51:41.015735 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:41.015739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:41.015948 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:41.016052 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:41.016496 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:41.017348 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:41.019263 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:41.019728 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:41.019940 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:41.021364 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:41.021374 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:41.021393 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:41.022072 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:41.022123 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:41.022144 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:41.022273 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:41.022451 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 20:51:41.022538 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 20:51:41.023738 25779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:41.089159 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:41.088492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42434)
0923 20:51:41.088842 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:41.088956 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260923 20:51:41.090606 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:41.089901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42450)
0923 20:51:41.090254 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:41.090357 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260923 20:51:41.148451 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25657
I20260923 20:51:41.169278 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25306
I20260923 20:51:41.170795 25367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:41.292594 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25306
I20260923 20:51:41.292673 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:41.291446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.129:33639 (local address 127.0.0.1:42454)
0923 20:51:41.291664 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 20:51:41.291824 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.129:33639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
2026-09-23T20:51:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3022 ms)
[----------] 6 tests from EncryptionPolicyTest (23197 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:41 dist-test-slave-dn29 krb5kdc[25835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:41 dist-test-slave-dn29 krb5kdc[25835](info): set up 2 sockets
Sep 23 20:51:41 dist-test-slave-dn29 krb5kdc[25835](info): commencing operation
krb5kdc: starting...
W20260923 20:51:42.460956 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:42 dist-test-slave-dn29 krb5kdc[25835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:42.752756 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:43.124413 25848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:43.124841 25848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:43.125036 25848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:43.133692 25848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:43.133893 25848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:43.134022 25848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:43.134136 25848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:43.134224 25848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:43.145280 25848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:43.146831 25848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:43.148350 25848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:43.164266 25853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:43.164857 25856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:43.164896 25854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:43.166409 25848 server_base.cc:1034] running on GCE node
I20260923 20:51:43.167375 25848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:43.169179 25848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:43.170495 25848 hybrid_clock.cc:648] HybridClock initialized: now 1790196703170361 us; error 115 us; skew 500 ppm
I20260923 20:51:43.170948 25848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:43 dist-test-slave-dn29 krb5kdc[25835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196703, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:43.174733 25848 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:43.176788 25848 webserver.cc:460] Webserver started at http://127.0.0.1:38283/ using document root <none> and password file <none>
I20260923 20:51:43.177628 25848 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:43.177750 25848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:43.178102 25848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:43.181536 25848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "b83b5698673c4dc3a39e35b49390faf8"
format_stamp: "Formatted at 2026-09-23 20:51:43 on dist-test-slave-dn29"
server_key: "09135045ab4b5c83f2690178b0a16bd0"
server_key_iv: "ace96d8fdcdb41740b65a8bc0ca704bf"
server_key_version: "encryptionkey@0"
I20260923 20:51:43.182411 25848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "b83b5698673c4dc3a39e35b49390faf8"
format_stamp: "Formatted at 2026-09-23 20:51:43 on dist-test-slave-dn29"
server_key: "09135045ab4b5c83f2690178b0a16bd0"
server_key_iv: "ace96d8fdcdb41740b65a8bc0ca704bf"
server_key_version: "encryptionkey@0"
I20260923 20:51:43.188719 25848 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260923 20:51:43.193172 25863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:43.195039 25848 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260923 20:51:43.195297 25848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "b83b5698673c4dc3a39e35b49390faf8"
format_stamp: "Formatted at 2026-09-23 20:51:43 on dist-test-slave-dn29"
server_key: "09135045ab4b5c83f2690178b0a16bd0"
server_key_iv: "ace96d8fdcdb41740b65a8bc0ca704bf"
server_key_version: "encryptionkey@0"
I20260923 20:51:43.195684 25848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:43.216094 25848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:43.221326 25848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:43.221766 25848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:43.244346 25848 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40427
I20260923 20:51:43.244382 25914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40427 every 8 connection(s)
I20260923 20:51:43.245911 25848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:43.250201 25915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:43.251946 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25848
I20260923 20:51:43.252194 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:43.253029 5406 external_mini_cluster.cc:1428] Setting key 23397a6f816176a9d8432b529a8b41fa
I20260923 20:51:43.266108 25915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8: Bootstrap starting.
Sep 23 20:51:43 dist-test-slave-dn29 krb5kdc[25835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 20:51:43.270377 25915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:43.271654 25915 log.cc:824] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:43.275722 25915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8: No bootstrap required, opened a new log
I20260923 20:51:43.280289 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:43.255243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56181 (local address 127.0.0.1:40427)
0923 20:51:43.256354 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:43.256368 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:43.256414 (+ 46us) server_negotiation.cc:407] Connection header received
0923 20:51:43.258224 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:43.258253 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:43.258754 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:43.259137 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:43.261342 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:43.262769 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:43.264722 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:43.265500 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:43.265748 (+ 248us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:43.271106 (+ 5358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:43.271115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:43.271132 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:43.271180 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:43.275183 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:43.276465 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:43.276473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:43.276480 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:43.276667 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:43.277175 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:43.277178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:43.277180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:43.277830 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:43.278092 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:43.278621 (+ 529us) server_negotiation.cc:299] Negotiation successful
0923 20:51:43.279066 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"thread_start_us":613,"threads_started":1}
I20260923 20:51:43.285073 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:43.285373 25915 raft_consensus.cc:348] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b5698673c4dc3a39e35b49390faf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260923 20:51:43.285853 25915 raft_consensus.cc:374] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:43.285964 25915 raft_consensus.cc:729] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b83b5698673c4dc3a39e35b49390faf8, State: Initialized, Role: FOLLOWER
I20260923 20:51:43.286666 25915 consensus_queue.cc:260] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b5698673c4dc3a39e35b49390faf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260923 20:51:43.286942 25915 raft_consensus.cc:388] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:43.287068 25915 raft_consensus.cc:482] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:43.287232 25915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:43.290967 25915 raft_consensus.cc:504] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b5698673c4dc3a39e35b49390faf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260923 20:51:43.291704 25915 leader_election.cc:302] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b83b5698673c4dc3a39e35b49390faf8; no voters:
I20260923 20:51:43.292507 25915 leader_election.cc:288] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:43.292671 25925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:43.293447 25925 raft_consensus.cc:686] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [term 1 LEADER]: Becoming Leader. State: Replica: b83b5698673c4dc3a39e35b49390faf8, State: Running, Role: LEADER
I20260923 20:51:43.294271 25925 consensus_queue.cc:237] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b5698673c4dc3a39e35b49390faf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260923 20:51:43.298169 25915 sys_catalog.cc:564] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:43.299187 25926 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b83b5698673c4dc3a39e35b49390faf8. Latest consensus state: current_term: 1 leader_uuid: "b83b5698673c4dc3a39e35b49390faf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b5698673c4dc3a39e35b49390faf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } } }
I20260923 20:51:43.299322 25927 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b83b5698673c4dc3a39e35b49390faf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b5698673c4dc3a39e35b49390faf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } } }
I20260923 20:51:43.299619 25926 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:43.299721 25927 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:43.303223 25929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:43.309763 25929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:43.316594 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:43.291472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51961 (local address 127.0.0.1:40427)
0923 20:51:43.291708 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:43.291718 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:43.292487 (+ 769us) server_negotiation.cc:407] Connection header received
0923 20:51:43.292731 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:43.292737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:43.292834 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:43.292936 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:43.295675 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:43.297531 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:43.299968 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:43.300888 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:43.301117 (+ 229us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:43.305061 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:43.305068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:43.305072 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:43.305118 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:43.310173 (+ 5055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:43.311759 (+ 1586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:43.311768 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:43.311774 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:43.311992 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:43.312762 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:43.312770 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:43.312777 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:43.313366 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:43.315439 (+ 2073us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:43.315713 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 20:51:43.316019 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 20:51:43.323117 25929 catalog_manager.cc:1347] Generated new cluster ID: 81f7850d6fa949d7b016f011b1681966
I20260923 20:51:43.323284 25929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:43.344514 25929 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:43.345918 25929 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:43.357280 25929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b83b5698673c4dc3a39e35b49390faf8: Generated new TSK 0
I20260923 20:51:43.358157 25929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:43.372987 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25848
I20260923 20:51:43.374984 25910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:43.480283 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25848
W20260923 20:51:43.500545 25833 connection.cc:535] client connection to 127.0.0.1:40427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40427): Connection reset by peer (error 104)
2026-09-23T20:51:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2179 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:43 dist-test-slave-dn29 krb5kdc[25947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:43 dist-test-slave-dn29 krb5kdc[25947](info): set up 2 sockets
Sep 23 20:51:43 dist-test-slave-dn29 krb5kdc[25947](info): commencing operation
krb5kdc: starting...
W20260923 20:51:44.644802 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:44 dist-test-slave-dn29 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:44.928977 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:45.291181 25960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:45.291610 25960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:45.291812 25960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:45.300230 25960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:45.300416 25960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:45.300511 25960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:45.300618 25960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:45.300706 25960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:45.311902 25960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45353
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:45.313405 25960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:45.314805 25960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:45.328280 25968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:45.328362 25965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:45.328313 25966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:45.329228 25960 server_base.cc:1034] running on GCE node
I20260923 20:51:45.330176 25960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:45.331985 25960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:45.333257 25960 hybrid_clock.cc:648] HybridClock initialized: now 1790196705333226 us; error 57 us; skew 500 ppm
I20260923 20:51:45.333882 25960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:45 dist-test-slave-dn29 krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:45.337936 25960 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:45.339649 25960 webserver.cc:460] Webserver started at http://127.0.0.1:36619/ using document root <none> and password file <none>
I20260923 20:51:45.340452 25960 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:45.340622 25960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:45.340955 25960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:45.344271 25960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "c3f913e6093a42688dd594b7ecf2bb8a"
format_stamp: "Formatted at 2026-09-23 20:51:45 on dist-test-slave-dn29"
server_key: "8448664801565bad9e3e102122ccc50d"
server_key_iv: "1d79a0e707face40420e904aa0cd9a09"
server_key_version: "encryptionkey@0"
I20260923 20:51:45.345100 25960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "c3f913e6093a42688dd594b7ecf2bb8a"
format_stamp: "Formatted at 2026-09-23 20:51:45 on dist-test-slave-dn29"
server_key: "8448664801565bad9e3e102122ccc50d"
server_key_iv: "1d79a0e707face40420e904aa0cd9a09"
server_key_version: "encryptionkey@0"
I20260923 20:51:45.351027 25960 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260923 20:51:45.355633 25975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:45.357369 25960 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:45.357678 25960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "c3f913e6093a42688dd594b7ecf2bb8a"
format_stamp: "Formatted at 2026-09-23 20:51:45 on dist-test-slave-dn29"
server_key: "8448664801565bad9e3e102122ccc50d"
server_key_iv: "1d79a0e707face40420e904aa0cd9a09"
server_key_version: "encryptionkey@0"
I20260923 20:51:45.357934 25960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:45.390144 25960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:45.396406 25960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:45.396754 25960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:45.417539 25960 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45353
I20260923 20:51:45.417622 26026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45353 every 8 connection(s)
I20260923 20:51:45.419035 25960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:45.423408 26027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:45.428119 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 25960
I20260923 20:51:45.428361 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:45.428922 5406 external_mini_cluster.cc:1428] Setting key ae624c622b7c7187b4143a0b08e6ef27
I20260923 20:51:45.436092 26027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a: Bootstrap starting.
I20260923 20:51:45.439459 26027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:45.440795 26027 log.cc:824] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:45.444837 26027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a: No bootstrap required, opened a new log
I20260923 20:51:45.445801 26030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:45.431151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57825 (local address 127.0.0.1:45353)
0923 20:51:45.432237 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:45.432256 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:45.432760 (+ 504us) server_negotiation.cc:407] Connection header received
0923 20:51:45.434892 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:45.434919 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:45.435441 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:45.435907 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:45.436519 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:45.437961 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:45.439526 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:45.440476 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:45.440732 (+ 256us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:45.442309 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:45.442324 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:45.442342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:45.443221 (+ 879us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:45.443333 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:45.443370 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:45.443568 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:45.444089 (+ 521us) server_negotiation.cc:299] Negotiation successful
0923 20:51:45.444292 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":501,"threads_started":1}
I20260923 20:51:45.451071 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:45.454227 26027 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260923 20:51:45.454710 26027 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:45.454849 26027 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f913e6093a42688dd594b7ecf2bb8a, State: Initialized, Role: FOLLOWER
I20260923 20:51:45.455557 26027 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260923 20:51:45.455852 26027 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:45.456094 26027 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:45.456322 26027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:45.459599 26027 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260923 20:51:45.460229 26027 leader_election.cc:302] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3f913e6093a42688dd594b7ecf2bb8a; no voters:
I20260923 20:51:45.461114 26027 leader_election.cc:288] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:45.461288 26037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:45.463660 26037 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [term 1 LEADER]: Becoming Leader. State: Replica: c3f913e6093a42688dd594b7ecf2bb8a, State: Running, Role: LEADER
I20260923 20:51:45.463845 26027 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:45.464459 26037 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260923 20:51:45.464731 26030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:45.456195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50025 (local address 127.0.0.1:45353)
0923 20:51:45.456405 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:45.456412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:45.456867 (+ 455us) server_negotiation.cc:407] Connection header received
0923 20:51:45.457057 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:45.457064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:45.457159 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:45.457282 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:45.457870 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:45.459015 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:45.460611 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:45.461518 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:45.462668 (+ 1150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:45.463244 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:45.463250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:45.463255 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:45.463279 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:45.463351 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:45.463373 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:45.464151 (+ 778us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:45.464229 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 20:51:45.464286 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260923 20:51:45.467973 26039 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3f913e6093a42688dd594b7ecf2bb8a. Latest consensus state: current_term: 1 leader_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } } }
I20260923 20:51:45.468097 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f913e6093a42688dd594b7ecf2bb8a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } } }
I20260923 20:51:45.468577 26039 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:45.468591 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:45.469586 26047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:45.476768 26047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:45.487304 26047 catalog_manager.cc:1347] Generated new cluster ID: 188ea9df03ba469ea3ef47a588d6c8e8
I20260923 20:51:45.487474 26047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:45.500268 26047 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:45.501997 26047 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:45.512434 26047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3f913e6093a42688dd594b7ecf2bb8a: Generated new TSK 0
I20260923 20:51:45.513455 26047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:45.527307 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25960
W20260923 20:51:45.529225 26022 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57825
W20260923 20:51:45.530076 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 25960: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:51:45.530280 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 25960
W20260923 20:51:45.544997 25945 connection.cc:535] client connection to 127.0.0.1:45353 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45353): Connection reset by peer (error 104)
2026-09-23T20:51:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:45 dist-test-slave-dn29 krb5kdc[26058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:45 dist-test-slave-dn29 krb5kdc[26058](info): set up 2 sockets
Sep 23 20:51:45 dist-test-slave-dn29 krb5kdc[26058](info): commencing operation
krb5kdc: starting...
W20260923 20:51:46.740866 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:46 dist-test-slave-dn29 krb5kdc[26058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:47.024442 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:47.386267 26071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:47.386627 26071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:47.386801 26071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:47.395073 26071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:47.395212 26071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:47.395298 26071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:47.395390 26071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:47.395457 26071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:47.407477 26071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:47.408936 26071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:47.410351 26071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:47.423410 26077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:47.423381 26079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:47.423763 26076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:47.424422 26071 server_base.cc:1034] running on GCE node
I20260923 20:51:47.425472 26071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:47.427205 26071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:47.428500 26071 hybrid_clock.cc:648] HybridClock initialized: now 1790196707428431 us; error 47 us; skew 500 ppm
I20260923 20:51:47.429013 26071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:47 dist-test-slave-dn29 krb5kdc[26058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:47.432814 26071 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:47.434579 26071 webserver.cc:460] Webserver started at http://127.0.0.1:42565/ using document root <none> and password file <none>
I20260923 20:51:47.435431 26071 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:47.435590 26071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:47.436012 26071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:47.439401 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "85fef4fc3359406a9c59972c1cd25080"
format_stamp: "Formatted at 2026-09-23 20:51:47 on dist-test-slave-dn29"
server_key: "340f99707ffffef8f8eac8e0aafffbe8"
server_key_iv: "48c958f9a003f5fc7f62eeb82d13ffb9"
server_key_version: "encryptionkey@0"
I20260923 20:51:47.440320 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "85fef4fc3359406a9c59972c1cd25080"
format_stamp: "Formatted at 2026-09-23 20:51:47 on dist-test-slave-dn29"
server_key: "340f99707ffffef8f8eac8e0aafffbe8"
server_key_iv: "48c958f9a003f5fc7f62eeb82d13ffb9"
server_key_version: "encryptionkey@0"
I20260923 20:51:47.446357 26071 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:51:47.450402 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:47.452214 26071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:47.452437 26071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "85fef4fc3359406a9c59972c1cd25080"
format_stamp: "Formatted at 2026-09-23 20:51:47 on dist-test-slave-dn29"
server_key: "340f99707ffffef8f8eac8e0aafffbe8"
server_key_iv: "48c958f9a003f5fc7f62eeb82d13ffb9"
server_key_version: "encryptionkey@0"
I20260923 20:51:47.452682 26071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:47.459245 26071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:47.464346 26071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:47.464682 26071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:47.485891 26071 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46749
I20260923 20:51:47.485939 26137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46749 every 8 connection(s)
I20260923 20:51:47.487493 26071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:47.492604 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:47.492774 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26071
I20260923 20:51:47.492940 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:47.493490 5406 external_mini_cluster.cc:1428] Setting key 1e25b35a55d5d4d2d2c0e2ca80d5d1c2
I20260923 20:51:47.502056 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:47.495508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51277 (local address 127.0.0.1:46749)
0923 20:51:47.496526 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:47.496538 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:47.496582 (+ 44us) server_negotiation.cc:407] Connection header received
0923 20:51:47.498176 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:47.498202 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:47.498602 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:47.499080 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:47.500317 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:47.500324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:47.500343 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:47.500370 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:47.500438 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:47.500463 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:47.500719 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:47.500865 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 20:51:47.501063 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":432,"threads_started":1}
I20260923 20:51:47.506652 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:47.507982 26138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080: Bootstrap starting.
I20260923 20:51:47.511662 26138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:47.513232 26138 log.cc:824] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:47.514274 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:47.511423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46693 (local address 127.0.0.1:46749)
0923 20:51:47.511659 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:47.511665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:47.512075 (+ 410us) server_negotiation.cc:407] Connection header received
0923 20:51:47.512228 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:47.512233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:47.512315 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:47.512432 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:47.513493 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:47.513499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:47.513509 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:47.513526 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:47.513571 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:47.513587 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:47.513745 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:47.513903 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 20:51:47.513976 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:51:47.516772 26138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080: No bootstrap required, opened a new log
I20260923 20:51:47.524626 26138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fef4fc3359406a9c59972c1cd25080" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46749 } }
I20260923 20:51:47.525055 26138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:47.525197 26138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85fef4fc3359406a9c59972c1cd25080, State: Initialized, Role: FOLLOWER
I20260923 20:51:47.525861 26138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fef4fc3359406a9c59972c1cd25080" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46749 } }
I20260923 20:51:47.526124 26138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:47.526291 26138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:47.526463 26138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:47.529203 26138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fef4fc3359406a9c59972c1cd25080" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46749 } }
I20260923 20:51:47.529824 26138 leader_election.cc:302] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85fef4fc3359406a9c59972c1cd25080; no voters:
I20260923 20:51:47.530472 26138 leader_election.cc:288] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:47.530834 26148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:47.531638 26148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [term 1 LEADER]: Becoming Leader. State: Replica: 85fef4fc3359406a9c59972c1cd25080, State: Running, Role: LEADER
I20260923 20:51:47.532384 26148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fef4fc3359406a9c59972c1cd25080" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46749 } }
I20260923 20:51:47.533082 26138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:47.535561 26150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85fef4fc3359406a9c59972c1cd25080. Latest consensus state: current_term: 1 leader_uuid: "85fef4fc3359406a9c59972c1cd25080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fef4fc3359406a9c59972c1cd25080" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46749 } } }
I20260923 20:51:47.536046 26150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:47.536187 26149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85fef4fc3359406a9c59972c1cd25080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fef4fc3359406a9c59972c1cd25080" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46749 } } }
I20260923 20:51:47.536574 26149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:47.541983 26154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:47.547693 26154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:47.558084 26154 catalog_manager.cc:1347] Generated new cluster ID: 5f31aa069370494cb49ee41ae7221a48
I20260923 20:51:47.558203 26154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:47.604526 26154 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:47.605051 26154 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:47.618798 26154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85fef4fc3359406a9c59972c1cd25080: Generated new TSK 0
I20260923 20:51:47.619729 26154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:47.627159 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26071
W20260923 20:51:47.628297 26133 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51277
W20260923 20:51:47.629050 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26071: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:51:47.629194 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26071
2026-09-23T20:51:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2102 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:47 dist-test-slave-dn29 krb5kdc[26169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:47 dist-test-slave-dn29 krb5kdc[26169](info): set up 2 sockets
Sep 23 20:51:47 dist-test-slave-dn29 krb5kdc[26169](info): commencing operation
krb5kdc: starting...
W20260923 20:51:48.784909 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:48 dist-test-slave-dn29 krb5kdc[26169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:49.064528 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46767
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:49.411186 26182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:49.411695 26182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:49.411890 26182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:49.420742 26182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:49.420908 26182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:49.421015 26182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:49.421130 26182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:49.421218 26182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:49.431545 26182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:49.432979 26182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:49.434402 26182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:49.446966 26187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:49.446934 26188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:49.446897 26190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:49.447850 26182 server_base.cc:1034] running on GCE node
I20260923 20:51:49.448791 26182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:49.450460 26182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:49.451905 26182 hybrid_clock.cc:648] HybridClock initialized: now 1790196709451730 us; error 63 us; skew 500 ppm
I20260923 20:51:49.452423 26182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:49 dist-test-slave-dn29 krb5kdc[26169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196709, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:49.456357 26182 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:49.458186 26182 webserver.cc:460] Webserver started at http://127.0.0.1:41141/ using document root <none> and password file <none>
I20260923 20:51:49.459003 26182 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:49.459156 26182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:49.459518 26182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:49.463068 26182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "da481c619738445dbe8322416258d95c"
format_stamp: "Formatted at 2026-09-23 20:51:49 on dist-test-slave-dn29"
server_key: "df478f1464760636d9f050ae98922613"
server_key_iv: "0b40a1904febb7cc0357414af8d49644"
server_key_version: "encryptionkey@0"
I20260923 20:51:49.464020 26182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "da481c619738445dbe8322416258d95c"
format_stamp: "Formatted at 2026-09-23 20:51:49 on dist-test-slave-dn29"
server_key: "df478f1464760636d9f050ae98922613"
server_key_iv: "0b40a1904febb7cc0357414af8d49644"
server_key_version: "encryptionkey@0"
I20260923 20:51:49.470635 26182 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260923 20:51:49.475227 26197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:49.477480 26182 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:49.477743 26182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "da481c619738445dbe8322416258d95c"
format_stamp: "Formatted at 2026-09-23 20:51:49 on dist-test-slave-dn29"
server_key: "df478f1464760636d9f050ae98922613"
server_key_iv: "0b40a1904febb7cc0357414af8d49644"
server_key_version: "encryptionkey@0"
I20260923 20:51:49.477958 26182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:49.509539 26182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:49.514595 26182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:49.514919 26182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:49.535254 26182 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46767
I20260923 20:51:49.535293 26248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46767 every 8 connection(s)
I20260923 20:51:49.536801 26182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:49.541047 26249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:49.544356 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26182
I20260923 20:51:49.544562 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:49.545060 5406 external_mini_cluster.cc:1428] Setting key f56da53e4e5c2c1cf3da7a84b2b80c39
I20260923 20:51:49.554346 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c: Bootstrap starting.
I20260923 20:51:49.558683 26249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:49.560191 26249 log.cc:824] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:51:49 dist-test-slave-dn29 krb5kdc[26169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 20:51:49.563723 26249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c: No bootstrap required, opened a new log
I20260923 20:51:49.572237 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:49.547231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42421 (local address 127.0.0.1:46767)
0923 20:51:49.548127 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:49.548140 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:49.548183 (+ 43us) server_negotiation.cc:407] Connection header received
0923 20:51:49.549913 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:49.549938 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:49.550431 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:49.550797 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:49.552792 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:49.554386 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:49.556371 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:49.557251 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:49.557422 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:49.562909 (+ 5487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:49.562918 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:49.562948 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:49.562994 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:49.567059 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:49.568070 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:49.568079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:49.568087 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:49.568301 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:49.568807 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:49.568811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:49.568813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:49.569502 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:49.569765 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:49.570301 (+ 536us) server_negotiation.cc:299] Negotiation successful
0923 20:51:49.570920 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":411,"threads_started":1}
I20260923 20:51:49.573177 26249 raft_consensus.cc:348] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da481c619738445dbe8322416258d95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46767 } }
I20260923 20:51:49.573645 26249 raft_consensus.cc:374] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:49.573770 26249 raft_consensus.cc:729] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da481c619738445dbe8322416258d95c, State: Initialized, Role: FOLLOWER
I20260923 20:51:49.574388 26249 consensus_queue.cc:260] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da481c619738445dbe8322416258d95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46767 } }
I20260923 20:51:49.574604 26249 raft_consensus.cc:388] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:49.574709 26249 raft_consensus.cc:482] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:49.574896 26249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:49.576617 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:49.577605 26249 raft_consensus.cc:504] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da481c619738445dbe8322416258d95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46767 } }
I20260923 20:51:49.578140 26249 leader_election.cc:302] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da481c619738445dbe8322416258d95c; no voters:
I20260923 20:51:49.578773 26249 leader_election.cc:288] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:49.578984 26254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:49.580812 26254 raft_consensus.cc:686] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [term 1 LEADER]: Becoming Leader. State: Replica: da481c619738445dbe8322416258d95c, State: Running, Role: LEADER
I20260923 20:51:49.581328 26249 sys_catalog.cc:564] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:49.581622 26254 consensus_queue.cc:237] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da481c619738445dbe8322416258d95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46767 } }
I20260923 20:51:49.590598 26255 sys_catalog.cc:455] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da481c619738445dbe8322416258d95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da481c619738445dbe8322416258d95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46767 } } }
I20260923 20:51:49.591187 26255 sys_catalog.cc:458] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:49.593230 26256 sys_catalog.cc:455] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [sys.catalog]: SysCatalogTable state changed. Reason: New leader da481c619738445dbe8322416258d95c. Latest consensus state: current_term: 1 leader_uuid: "da481c619738445dbe8322416258d95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da481c619738445dbe8322416258d95c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46767 } } }
I20260923 20:51:49.593632 26256 sys_catalog.cc:458] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:49.593791 26268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:49.599455 26268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:49.608466 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:49.583880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46141 (local address 127.0.0.1:46767)
0923 20:51:49.584130 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:49.584141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:49.584506 (+ 365us) server_negotiation.cc:407] Connection header received
0923 20:51:49.584784 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:49.584792 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:49.584909 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:49.585041 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:49.588117 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:49.589662 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:49.592434 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:49.593438 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:49.593798 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:49.597593 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:49.597604 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:49.597620 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:49.597678 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:49.603227 (+ 5549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:49.604486 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:49.604492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:49.604496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:49.604688 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:49.605310 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:49.605315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:49.605318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:49.605787 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:49.606077 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:49.607535 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0923 20:51:49.607911 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 20:51:49.613785 26268 catalog_manager.cc:1347] Generated new cluster ID: 73d19093cb454423b0b58d8e7e654850
I20260923 20:51:49.613910 26268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:49.631577 26268 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:49.633399 26268 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:49.647377 26268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da481c619738445dbe8322416258d95c: Generated new TSK 0
I20260923 20:51:49.648486 26268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:49.657115 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26182
I20260923 20:51:49.658834 26244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:49.755309 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26182
2026-09-23T20:51:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2127 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:49 dist-test-slave-dn29 krb5kdc[26282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:49 dist-test-slave-dn29 krb5kdc[26282](info): set up 2 sockets
Sep 23 20:51:49 dist-test-slave-dn29 krb5kdc[26282](info): commencing operation
krb5kdc: starting...
W20260923 20:51:50.968716 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:51 dist-test-slave-dn29 krb5kdc[26282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:51.260196 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:51.616618 26295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:51.616926 26295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:51.617019 26295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:51.624570 26295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:51.624713 26295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:51.624759 26295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:51.624814 26295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:51.624858 26295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:51.635308 26295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:51.636761 26295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:51.638118 26295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:51.651176 26300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:51.651141 26303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:51.651119 26301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:51.652235 26295 server_base.cc:1034] running on GCE node
I20260923 20:51:51.653267 26295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:51.654958 26295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:51.656224 26295 hybrid_clock.cc:648] HybridClock initialized: now 1790196711656169 us; error 42 us; skew 500 ppm
I20260923 20:51:51.656719 26295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:51 dist-test-slave-dn29 krb5kdc[26282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:51.668967 26295 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:51.670837 26295 webserver.cc:460] Webserver started at http://127.0.0.1:44913/ using document root <none> and password file <none>
I20260923 20:51:51.671674 26295 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:51.671866 26295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:51.672209 26295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:51.675387 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "8ef2ec228dec4376ac5e4b0e1afee666"
format_stamp: "Formatted at 2026-09-23 20:51:51 on dist-test-slave-dn29"
server_key: "aa6ee991edcbb4aebe855c1fc9f3d1e4"
server_key_iv: "4362cfe9635b65b0ee025b49d847edd0"
server_key_version: "encryptionkey@0"
I20260923 20:51:51.676272 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "8ef2ec228dec4376ac5e4b0e1afee666"
format_stamp: "Formatted at 2026-09-23 20:51:51 on dist-test-slave-dn29"
server_key: "aa6ee991edcbb4aebe855c1fc9f3d1e4"
server_key_iv: "4362cfe9635b65b0ee025b49d847edd0"
server_key_version: "encryptionkey@0"
I20260923 20:51:51.682242 26295 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260923 20:51:51.686389 26310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:51.688195 26295 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:51:51.688469 26295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "8ef2ec228dec4376ac5e4b0e1afee666"
format_stamp: "Formatted at 2026-09-23 20:51:51 on dist-test-slave-dn29"
server_key: "aa6ee991edcbb4aebe855c1fc9f3d1e4"
server_key_iv: "4362cfe9635b65b0ee025b49d847edd0"
server_key_version: "encryptionkey@0"
I20260923 20:51:51.688707 26295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:51.701006 26295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:51.705665 26295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:51.706010 26295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:51.726562 26295 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39965
I20260923 20:51:51.726606 26361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39965 every 8 connection(s)
I20260923 20:51:51.728116 26295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:51.728597 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26295
I20260923 20:51:51.728794 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:51.729454 5406 external_mini_cluster.cc:1428] Setting key 8044c3bbc7e19e8494af7635e3d9fbce
I20260923 20:51:51.732929 26362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:51.745493 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:51.731444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46335 (local address 127.0.0.1:39965)
0923 20:51:51.732321 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:51.732337 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:51.732386 (+ 49us) server_negotiation.cc:407] Connection header received
0923 20:51:51.734741 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:51.734769 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:51.735264 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:51.735709 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:51.736439 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:51.738471 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:51.740144 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:51.741166 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:51.741407 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:51.742518 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:51.742530 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:51.742550 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:51.743316 (+ 766us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:51.743415 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:51.743451 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:51.743569 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:51.743764 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 20:51:51.744093 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":407,"threads_started":1}
I20260923 20:51:51.747438 26362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666: Bootstrap starting.
I20260923 20:51:51.749509 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:51.751101 26362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:51.752279 26362 log.cc:824] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:51.755518 26362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666: No bootstrap required, opened a new log
I20260923 20:51:51.761307 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:51.754087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40723 (local address 127.0.0.1:39965)
0923 20:51:51.754320 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:51.754330 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:51.754896 (+ 566us) server_negotiation.cc:407] Connection header received
0923 20:51:51.755046 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:51.755052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:51.755127 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:51.755248 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:51.755867 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:51.757007 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:51.758297 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:51.759011 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:51.759147 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:51.760366 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:51.760373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:51.760379 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:51.760407 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:51.760480 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:51.760506 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:51.760605 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:51.760818 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 20:51:51.760923 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 20:51:51.765015 26362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260923 20:51:51.765391 26362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:51.765455 26362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef2ec228dec4376ac5e4b0e1afee666, State: Initialized, Role: FOLLOWER
I20260923 20:51:51.766094 26362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260923 20:51:51.766397 26362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:51.766572 26362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:51.766769 26362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:51.769022 26362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260923 20:51:51.769551 26362 leader_election.cc:302] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ef2ec228dec4376ac5e4b0e1afee666; no voters:
I20260923 20:51:51.770242 26362 leader_election.cc:288] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:51.770388 26372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:51.771243 26372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [term 1 LEADER]: Becoming Leader. State: Replica: 8ef2ec228dec4376ac5e4b0e1afee666, State: Running, Role: LEADER
I20260923 20:51:51.771898 26372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260923 20:51:51.772568 26362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:51.774976 26374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ef2ec228dec4376ac5e4b0e1afee666. Latest consensus state: current_term: 1 leader_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } } }
I20260923 20:51:51.775449 26374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:51.775978 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ef2ec228dec4376ac5e4b0e1afee666" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } } }
I20260923 20:51:51.776361 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:51.778627 26380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:51.784004 26380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:51.794746 26380 catalog_manager.cc:1347] Generated new cluster ID: bb33cadb952e446794f31c80291763a1
I20260923 20:51:51.794878 26380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:51.809303 26380 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:51.810995 26380 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:51.820705 26380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ef2ec228dec4376ac5e4b0e1afee666: Generated new TSK 0
I20260923 20:51:51.821671 26380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:51.832854 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26295
W20260923 20:51:51.834347 26357 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46335
W20260923 20:51:51.835228 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26295: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:51:51.835391 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26295
2026-09-23T20:51:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2078 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:51 dist-test-slave-dn29 krb5kdc[26393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:51 dist-test-slave-dn29 krb5kdc[26393](info): set up 2 sockets
Sep 23 20:51:51 dist-test-slave-dn29 krb5kdc[26393](info): commencing operation
krb5kdc: starting...
W20260923 20:51:52.989203 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:53 dist-test-slave-dn29 krb5kdc[26393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:53.283506 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:53.691851 26406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:53.692229 26406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:53.692405 26406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:53.699775 26406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:53.699987 26406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:53.700101 26406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:53.700217 26406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:53.700311 26406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:53.710546 26406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:53.712049 26406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:53.713429 26406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:53.726253 26411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:53.726310 26412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:53.726497 26414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:53.727066 26406 server_base.cc:1034] running on GCE node
I20260923 20:51:53.727968 26406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:53.729519 26406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:53.730818 26406 hybrid_clock.cc:648] HybridClock initialized: now 1790196713730760 us; error 46 us; skew 500 ppm
I20260923 20:51:53.731225 26406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:53 dist-test-slave-dn29 krb5kdc[26393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:53.734784 26406 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:53.736477 26406 webserver.cc:460] Webserver started at http://127.0.0.1:38235/ using document root <none> and password file <none>
I20260923 20:51:53.737205 26406 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:53.737360 26406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:53.737670 26406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:53.740790 26406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "094eb84ffef9435b8102c8bf7985736c"
format_stamp: "Formatted at 2026-09-23 20:51:53 on dist-test-slave-dn29"
server_key: "546d7b24dc90b82b7571928c1142e882"
server_key_iv: "31455bd52a159580955adaa2e728c578"
server_key_version: "encryptionkey@0"
I20260923 20:51:53.741564 26406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "094eb84ffef9435b8102c8bf7985736c"
format_stamp: "Formatted at 2026-09-23 20:51:53 on dist-test-slave-dn29"
server_key: "546d7b24dc90b82b7571928c1142e882"
server_key_iv: "31455bd52a159580955adaa2e728c578"
server_key_version: "encryptionkey@0"
I20260923 20:51:53.746981 26406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:51:53.750687 26421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:53.752238 26406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:51:53.752430 26406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "094eb84ffef9435b8102c8bf7985736c"
format_stamp: "Formatted at 2026-09-23 20:51:53 on dist-test-slave-dn29"
server_key: "546d7b24dc90b82b7571928c1142e882"
server_key_iv: "31455bd52a159580955adaa2e728c578"
server_key_version: "encryptionkey@0"
I20260923 20:51:53.752722 26406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:53.758865 26406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:53.763743 26406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:53.764163 26406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:53.786054 26406 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35761
I20260923 20:51:53.786088 26472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35761 every 8 connection(s)
I20260923 20:51:53.787555 26406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:53.791440 26473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:53.793192 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26406
I20260923 20:51:53.793376 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:53.793947 5406 external_mini_cluster.cc:1428] Setting key 7e47510ef6ba92015f5bb8a63b68c2a8
I20260923 20:51:53.801910 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:53.795994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58377 (local address 127.0.0.1:35761)
0923 20:51:53.796819 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:53.796832 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:53.796878 (+ 46us) server_negotiation.cc:407] Connection header received
0923 20:51:53.798490 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:53.798513 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:53.798884 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:53.799283 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:53.800357 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:53.800363 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:53.800376 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:53.800390 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:53.800442 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:53.800460 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:53.800706 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:53.800852 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 20:51:53.801021 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":345,"threads_started":1}
I20260923 20:51:53.805384 26473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c: Bootstrap starting.
I20260923 20:51:53.805888 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:53.808697 26473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:53.809903 26473 log.cc:824] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:53.813334 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:53.810764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42907 (local address 127.0.0.1:35761)
0923 20:51:53.811002 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:53.811009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:53.811318 (+ 309us) server_negotiation.cc:407] Connection header received
0923 20:51:53.811520 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:53.811532 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:53.811583 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:53.811703 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:53.812673 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:53.812678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:53.812690 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:53.812708 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:53.812753 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:53.812769 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:53.812893 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:53.812996 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 20:51:53.813058 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:51:53.813447 26473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c: No bootstrap required, opened a new log
I20260923 20:51:53.822762 26473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094eb84ffef9435b8102c8bf7985736c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260923 20:51:53.823328 26473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:53.823542 26473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094eb84ffef9435b8102c8bf7985736c, State: Initialized, Role: FOLLOWER
I20260923 20:51:53.824260 26473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094eb84ffef9435b8102c8bf7985736c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260923 20:51:53.824501 26473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:53.824630 26473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:53.824767 26473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:53.827183 26473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094eb84ffef9435b8102c8bf7985736c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260923 20:51:53.827705 26473 leader_election.cc:302] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 094eb84ffef9435b8102c8bf7985736c; no voters:
I20260923 20:51:53.828523 26473 leader_election.cc:288] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:53.828696 26483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:53.830605 26473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:53.832180 26483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [term 1 LEADER]: Becoming Leader. State: Replica: 094eb84ffef9435b8102c8bf7985736c, State: Running, Role: LEADER
I20260923 20:51:53.832799 26483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094eb84ffef9435b8102c8bf7985736c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260923 20:51:53.836180 26485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 094eb84ffef9435b8102c8bf7985736c. Latest consensus state: current_term: 1 leader_uuid: "094eb84ffef9435b8102c8bf7985736c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094eb84ffef9435b8102c8bf7985736c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } } }
I20260923 20:51:53.836697 26485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:53.837216 26484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "094eb84ffef9435b8102c8bf7985736c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094eb84ffef9435b8102c8bf7985736c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } } }
I20260923 20:51:53.837503 26484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:53.838971 26494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:53.844662 26494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:53.855413 26494 catalog_manager.cc:1347] Generated new cluster ID: d490a618c7fd49e7b80326ed786b9d50
I20260923 20:51:53.855545 26494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:53.871645 26494 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:53.872143 26494 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:53.892228 26494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 094eb84ffef9435b8102c8bf7985736c: Generated new TSK 0
I20260923 20:51:53.893169 26494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:53.902232 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26406
W20260923 20:51:53.903575 26468 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58377
W20260923 20:51:53.904449 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26406: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:51:53.904624 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26406
2026-09-23T20:51:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:54 dist-test-slave-dn29 krb5kdc[26504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:54 dist-test-slave-dn29 krb5kdc[26504](info): set up 2 sockets
Sep 23 20:51:54 dist-test-slave-dn29 krb5kdc[26504](info): commencing operation
krb5kdc: starting...
W20260923 20:51:55.056695 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.006s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:55 dist-test-slave-dn29 krb5kdc[26504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:55.344477 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:55.691565 26517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:55.691998 26517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:55.692174 26517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:55.700646 26517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:55.700839 26517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:55.700963 26517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:55.701054 26517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:55.701124 26517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:55.711500 26517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:55.713006 26517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:55.714440 26517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:55.727424 26523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:55.727386 26522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:55.728014 26525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:55.728618 26517 server_base.cc:1034] running on GCE node
I20260923 20:51:55.729537 26517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:55.731354 26517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:55.732613 26517 hybrid_clock.cc:648] HybridClock initialized: now 1790196715732572 us; error 34 us; skew 500 ppm
I20260923 20:51:55.733093 26517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:55 dist-test-slave-dn29 krb5kdc[26504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:55.736617 26517 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:55.738317 26517 webserver.cc:460] Webserver started at http://127.0.0.1:37195/ using document root <none> and password file <none>
I20260923 20:51:55.739063 26517 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:55.739225 26517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:55.739578 26517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:55.742923 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "820449aca0634b97aedc5f3825a61d1f"
format_stamp: "Formatted at 2026-09-23 20:51:55 on dist-test-slave-dn29"
server_key: "c80c5091a4b1c4148c039b97333c532c"
server_key_iv: "57385c8504ec4c7d586cd7ad2f50c525"
server_key_version: "encryptionkey@0"
I20260923 20:51:55.743764 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "820449aca0634b97aedc5f3825a61d1f"
format_stamp: "Formatted at 2026-09-23 20:51:55 on dist-test-slave-dn29"
server_key: "c80c5091a4b1c4148c039b97333c532c"
server_key_iv: "57385c8504ec4c7d586cd7ad2f50c525"
server_key_version: "encryptionkey@0"
I20260923 20:51:55.749536 26517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260923 20:51:55.753489 26532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:55.755076 26517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:51:55.755309 26517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "820449aca0634b97aedc5f3825a61d1f"
format_stamp: "Formatted at 2026-09-23 20:51:55 on dist-test-slave-dn29"
server_key: "c80c5091a4b1c4148c039b97333c532c"
server_key_iv: "57385c8504ec4c7d586cd7ad2f50c525"
server_key_version: "encryptionkey@0"
I20260923 20:51:55.755534 26517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:55.776548 26517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:55.781769 26517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:55.782153 26517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:55.803445 26517 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34273
I20260923 20:51:55.803465 26583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34273 every 8 connection(s)
I20260923 20:51:55.804953 26517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:55.808959 26584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:55.811970 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26517
I20260923 20:51:55.812148 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:55.812592 5406 external_mini_cluster.cc:1428] Setting key e2267abb8e9bee3ea629b1bd19167906
I20260923 20:51:55.821697 26584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f: Bootstrap starting.
I20260923 20:51:55.825681 26584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:55.827071 26584 log.cc:824] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:51:55 dist-test-slave-dn29 krb5kdc[26504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 20:51:55.831123 26584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f: No bootstrap required, opened a new log
I20260923 20:51:55.839001 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:55.814779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:35541 (local address 127.0.0.1:34273)
0923 20:51:55.815580 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:55.815593 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:55.815635 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:51:55.817474 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:55.817499 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:55.817940 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:55.818318 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:55.820150 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:55.821513 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:55.823279 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:55.824102 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:55.824257 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:55.830032 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:55.830042 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:55.830058 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:55.830100 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:55.833972 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:55.835138 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:55.835148 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:55.835159 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:55.835346 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:55.835993 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:55.835998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:55.836000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:55.836573 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:55.836913 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:55.837373 (+ 460us) server_negotiation.cc:299] Negotiation successful
0923 20:51:55.837805 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":358,"threads_started":1}
I20260923 20:51:55.841451 26584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820449aca0634b97aedc5f3825a61d1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34273 } }
I20260923 20:51:55.841944 26584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:55.842069 26584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820449aca0634b97aedc5f3825a61d1f, State: Initialized, Role: FOLLOWER
I20260923 20:51:55.842782 26584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820449aca0634b97aedc5f3825a61d1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34273 } }
I20260923 20:51:55.843073 26584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:55.843231 26584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:55.843442 26584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:55.843745 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:55.845773 26584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820449aca0634b97aedc5f3825a61d1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34273 } }
I20260923 20:51:55.846314 26584 leader_election.cc:302] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 820449aca0634b97aedc5f3825a61d1f; no voters:
I20260923 20:51:55.847072 26584 leader_election.cc:288] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:55.847160 26592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:55.848165 26592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [term 1 LEADER]: Becoming Leader. State: Replica: 820449aca0634b97aedc5f3825a61d1f, State: Running, Role: LEADER
I20260923 20:51:55.848863 26592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820449aca0634b97aedc5f3825a61d1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34273 } }
I20260923 20:51:55.849654 26584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:55.852097 26595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 820449aca0634b97aedc5f3825a61d1f. Latest consensus state: current_term: 1 leader_uuid: "820449aca0634b97aedc5f3825a61d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820449aca0634b97aedc5f3825a61d1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34273 } } }
I20260923 20:51:55.852614 26595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:55.853829 26594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "820449aca0634b97aedc5f3825a61d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "820449aca0634b97aedc5f3825a61d1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34273 } } }
I20260923 20:51:55.854172 26594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:55.855409 26599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:55.861195 26599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:55.869758 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:55.848955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:60957 (local address 127.0.0.1:34273)
0923 20:51:55.849114 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:55.849121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:55.849342 (+ 221us) server_negotiation.cc:407] Connection header received
0923 20:51:55.849608 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:55.849614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:55.849695 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:55.849791 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:55.851598 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:55.852874 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:55.854625 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:55.855420 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:55.855564 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:55.859714 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:55.859725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:55.859733 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:51:55.859808 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:55.864961 (+ 5153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:55.866172 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:55.866178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:55.866183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:55.866338 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:51:55.866965 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:51:55.866969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:51:55.866972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:51:55.867453 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:55.867742 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:55.869068 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0923 20:51:55.869254 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 20:51:55.878296 26599 catalog_manager.cc:1347] Generated new cluster ID: 3fed97a4827f41aaa5dd901690bb1514
I20260923 20:51:55.878433 26599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:55.888168 26599 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:55.889582 26599 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:55.898952 26599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 820449aca0634b97aedc5f3825a61d1f: Generated new TSK 0
I20260923 20:51:55.900056 26599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:55.909098 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26517
I20260923 20:51:55.910280 26579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:51:56.002134 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26517
W20260923 20:51:56.022032 26502 connection.cc:535] client connection to 127.0.0.1:34273 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34273): Connection reset by peer (error 104)
2026-09-23T20:51:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2102 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:56 dist-test-slave-dn29 krb5kdc[26616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:56 dist-test-slave-dn29 krb5kdc[26616](info): set up 2 sockets
Sep 23 20:51:56 dist-test-slave-dn29 krb5kdc[26616](info): commencing operation
krb5kdc: starting...
W20260923 20:51:57.144542 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:57 dist-test-slave-dn29 krb5kdc[26616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:57.433439 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:57.785979 26629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:57.786294 26629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:57.786387 26629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:57.793871 26629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:57.793992 26629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:57.794039 26629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:57.794100 26629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:57.794144 26629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:57.804147 26629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:57.805490 26629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:57.806795 26629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:57.819970 26634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:57.819907 26637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:57.819854 26635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:57.821497 26629 server_base.cc:1034] running on GCE node
I20260923 20:51:57.822469 26629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:57.824272 26629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:57.825526 26629 hybrid_clock.cc:648] HybridClock initialized: now 1790196717825465 us; error 45 us; skew 500 ppm
I20260923 20:51:57.825955 26629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:57 dist-test-slave-dn29 krb5kdc[26616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:57.834263 26629 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:57.836099 26629 webserver.cc:460] Webserver started at http://127.0.0.1:38133/ using document root <none> and password file <none>
I20260923 20:51:57.836933 26629 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:57.837100 26629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:57.837445 26629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:57.840890 26629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "0606932e22594d64a684c3c511e59a37"
format_stamp: "Formatted at 2026-09-23 20:51:57 on dist-test-slave-dn29"
server_key: "ff5e17a3bc96917a71f77248e71275ce"
server_key_iv: "bd7c2e7ae85e8cf6034d6f369a80a9cf"
server_key_version: "encryptionkey@0"
I20260923 20:51:57.841771 26629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "0606932e22594d64a684c3c511e59a37"
format_stamp: "Formatted at 2026-09-23 20:51:57 on dist-test-slave-dn29"
server_key: "ff5e17a3bc96917a71f77248e71275ce"
server_key_iv: "bd7c2e7ae85e8cf6034d6f369a80a9cf"
server_key_version: "encryptionkey@0"
I20260923 20:51:57.848353 26629 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:51:57.852563 26644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:57.854188 26629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:51:57.854503 26629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "0606932e22594d64a684c3c511e59a37"
format_stamp: "Formatted at 2026-09-23 20:51:57 on dist-test-slave-dn29"
server_key: "ff5e17a3bc96917a71f77248e71275ce"
server_key_iv: "bd7c2e7ae85e8cf6034d6f369a80a9cf"
server_key_version: "encryptionkey@0"
I20260923 20:51:57.854759 26629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:57.886119 26629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:57.890818 26629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:57.891148 26629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:57.911227 26629 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46857
I20260923 20:51:57.911296 26695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46857 every 8 connection(s)
I20260923 20:51:57.912740 26629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:51:57.914049 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26629
I20260923 20:51:57.914252 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:51:57.914908 5406 external_mini_cluster.cc:1428] Setting key d5743d8996bcbb505bdd5862cd385fe4
I20260923 20:51:57.917671 26696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:51:57.928808 26696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37: Bootstrap starting.
I20260923 20:51:57.930203 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:57.916923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:58843 (local address 127.0.0.1:46857)
0923 20:51:57.917896 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:57.917911 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:57.917960 (+ 49us) server_negotiation.cc:407] Connection header received
0923 20:51:57.919851 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:57.919889 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:57.920359 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:57.920816 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:57.921413 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:57.922910 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:57.924517 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:57.925274 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:57.925538 (+ 264us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:57.926983 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:57.926992 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:57.927009 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:57.927611 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:57.927696 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:57.927723 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:57.927950 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:57.928238 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 20:51:57.928455 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":351,"threads_started":1}
I20260923 20:51:57.932753 26696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37: Neither blocks nor log segments found. Creating new log.
I20260923 20:51:57.933996 26696 log.cc:824] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37: Log is configured to *not* fsync() on all Append() calls
I20260923 20:51:57.934537 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:51:57.937058 26696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37: No bootstrap required, opened a new log
I20260923 20:51:57.945497 26696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0606932e22594d64a684c3c511e59a37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46857 } }
I20260923 20:51:57.945990 26696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:51:57.946125 26696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0606932e22594d64a684c3c511e59a37, State: Initialized, Role: FOLLOWER
I20260923 20:51:57.946658 26699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:51:57.939323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:33569 (local address 127.0.0.1:46857)
0923 20:51:57.939552 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 20:51:57.939561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:51:57.940093 (+ 532us) server_negotiation.cc:407] Connection header received
0923 20:51:57.940308 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:51:57.940324 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:51:57.940391 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:51:57.940518 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:51:57.941070 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:57.941980 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:57.943553 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:51:57.944207 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:51:57.944385 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:51:57.945664 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:51:57.945669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:51:57.945674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:51:57.945696 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:51:57.945758 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:51:57.945776 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:51:57.945961 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0923 20:51:57.946182 (+ 221us) server_negotiation.cc:299] Negotiation successful
0923 20:51:57.946245 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 20:51:57.946928 26696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0606932e22594d64a684c3c511e59a37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46857 } }
I20260923 20:51:57.947218 26696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:51:57.947374 26696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:51:57.947561 26696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:51:57.950047 26696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0606932e22594d64a684c3c511e59a37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46857 } }
I20260923 20:51:57.950625 26696 leader_election.cc:302] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0606932e22594d64a684c3c511e59a37; no voters:
I20260923 20:51:57.951275 26696 leader_election.cc:288] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:51:57.951442 26706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:51:57.952723 26706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [term 1 LEADER]: Becoming Leader. State: Replica: 0606932e22594d64a684c3c511e59a37, State: Running, Role: LEADER
I20260923 20:51:57.953348 26706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0606932e22594d64a684c3c511e59a37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46857 } }
I20260923 20:51:57.953584 26696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:51:57.956660 26707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0606932e22594d64a684c3c511e59a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0606932e22594d64a684c3c511e59a37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46857 } } }
I20260923 20:51:57.957103 26707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:57.957561 26708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0606932e22594d64a684c3c511e59a37. Latest consensus state: current_term: 1 leader_uuid: "0606932e22594d64a684c3c511e59a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0606932e22594d64a684c3c511e59a37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46857 } } }
I20260923 20:51:57.957815 26708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37 [sys.catalog]: This master's current role is: LEADER
I20260923 20:51:57.965590 26715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:51:57.971167 26715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:51:57.981837 26715 catalog_manager.cc:1347] Generated new cluster ID: 393059041735433880f04f495ed2f62e
I20260923 20:51:57.981980 26715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:51:57.999630 26715 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:51:58.001701 26715 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:51:58.011385 26715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0606932e22594d64a684c3c511e59a37: Generated new TSK 0
I20260923 20:51:58.012687 26715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:51:58.023418 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26629
W20260923 20:51:58.025341 26691 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.74:58843
W20260923 20:51:58.026314 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26629: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:51:58.026486 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26629
W20260923 20:51:58.044652 26614 connection.cc:535] client connection to 127.0.0.1:46857 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46857): Connection reset by peer (error 104)
2026-09-23T20:51:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:51:58 dist-test-slave-dn29 krb5kdc[26727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:51:58 dist-test-slave-dn29 krb5kdc[26727](info): set up 2 sockets
Sep 23 20:51:58 dist-test-slave-dn29 krb5kdc[26727](info): commencing operation
krb5kdc: starting...
W20260923 20:51:59.248616 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:51:59 dist-test-slave-dn29 krb5kdc[26727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:51:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:51:59.536705 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38089
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:51:59.893744 26740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:51:59.894109 26740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:51:59.894310 26740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:51:59.902963 26740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:51:59.903105 26740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:51:59.903232 26740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:51:59.903364 26740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:51:59.903463 26740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:51:59.915130 26740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:51:59.916764 26740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:51:59.918567 26740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:51:59.936563 26746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:59.936645 26745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:51:59.936666 26748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:51:59.937872 26740 server_base.cc:1034] running on GCE node
I20260923 20:51:59.938848 26740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:51:59.940626 26740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:51:59.941910 26740 hybrid_clock.cc:648] HybridClock initialized: now 1790196719941861 us; error 37 us; skew 500 ppm
I20260923 20:51:59.942324 26740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:51:59 dist-test-slave-dn29 krb5kdc[26727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196719, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:51:59.946142 26740 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:51:59.948184 26740 webserver.cc:460] Webserver started at http://127.0.0.1:43583/ using document root <none> and password file <none>
I20260923 20:51:59.949060 26740 fs_manager.cc:359] Metadata directory not provided
I20260923 20:51:59.949241 26740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:51:59.949601 26740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:51:59.953179 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "5ba29d4158b540d9b7f0b75854eb49da"
format_stamp: "Formatted at 2026-09-23 20:51:59 on dist-test-slave-dn29"
server_key: "363e1a62da7bdbd81ecabbfa7c366f66"
server_key_iv: "50071dfa3cc2a9ecb48ffa9429540164"
server_key_version: "encryptionkey@0"
I20260923 20:51:59.954077 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "5ba29d4158b540d9b7f0b75854eb49da"
format_stamp: "Formatted at 2026-09-23 20:51:59 on dist-test-slave-dn29"
server_key: "363e1a62da7bdbd81ecabbfa7c366f66"
server_key_iv: "50071dfa3cc2a9ecb48ffa9429540164"
server_key_version: "encryptionkey@0"
I20260923 20:51:59.960073 26740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260923 20:51:59.964244 26755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:51:59.965837 26740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:51:59.966041 26740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "5ba29d4158b540d9b7f0b75854eb49da"
format_stamp: "Formatted at 2026-09-23 20:51:59 on dist-test-slave-dn29"
server_key: "363e1a62da7bdbd81ecabbfa7c366f66"
server_key_iv: "50071dfa3cc2a9ecb48ffa9429540164"
server_key_version: "encryptionkey@0"
I20260923 20:51:59.966291 26740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:51:59.973133 26740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:51:59.977841 26740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:51:59.978178 26740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:51:59.999014 26740 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38089
I20260923 20:51:59.999346 26806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38089 every 8 connection(s)
I20260923 20:52:00.002453 26740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:00.004206 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26740
I20260923 20:52:00.004400 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:00.004980 5406 external_mini_cluster.cc:1428] Setting key 1c143048f051f1f234e091d0561c454c
I20260923 20:52:00.008086 26807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:00.014935 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:00.007134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:55679 (local address 127.0.0.1:38089)
0923 20:52:00.008083 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:00.008100 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:00.008162 (+ 62us) server_negotiation.cc:407] Connection header received
0923 20:52:00.010556 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:00.010588 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:00.011072 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:00.011605 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:00.012768 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:00.012777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:00.012797 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:00.012817 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:00.012929 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:00.012957 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:00.013195 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:00.013336 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 20:52:00.013556 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":352,"threads_started":1}
I20260923 20:52:00.019915 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:00.022370 26807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da: Bootstrap starting.
I20260923 20:52:00.026084 26807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:00.027693 26807 log.cc:824] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:00.029397 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:00.025914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:45007 (local address 127.0.0.1:38089)
0923 20:52:00.026148 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:00.026156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:00.026816 (+ 660us) server_negotiation.cc:407] Connection header received
0923 20:52:00.026998 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:00.027002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:00.027056 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:00.027150 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:00.028433 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:00.028441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:00.028449 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:00.028477 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:00.028554 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:00.028577 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:00.028727 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:00.028978 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 20:52:00.029051 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 20:52:00.031586 26807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da: No bootstrap required, opened a new log
I20260923 20:52:00.039193 26807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba29d4158b540d9b7f0b75854eb49da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38089 } }
I20260923 20:52:00.039660 26807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:00.039798 26807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba29d4158b540d9b7f0b75854eb49da, State: Initialized, Role: FOLLOWER
I20260923 20:52:00.040512 26807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba29d4158b540d9b7f0b75854eb49da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38089 } }
I20260923 20:52:00.040767 26807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:00.040930 26807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:00.041127 26807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:00.043493 26807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba29d4158b540d9b7f0b75854eb49da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38089 } }
I20260923 20:52:00.044150 26807 leader_election.cc:302] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ba29d4158b540d9b7f0b75854eb49da; no voters:
I20260923 20:52:00.045049 26807 leader_election.cc:288] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:00.045512 26817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:00.047070 26817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [term 1 LEADER]: Becoming Leader. State: Replica: 5ba29d4158b540d9b7f0b75854eb49da, State: Running, Role: LEADER
I20260923 20:52:00.047737 26817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba29d4158b540d9b7f0b75854eb49da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38089 } }
I20260923 20:52:00.047914 26807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:00.050640 26819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ba29d4158b540d9b7f0b75854eb49da. Latest consensus state: current_term: 1 leader_uuid: "5ba29d4158b540d9b7f0b75854eb49da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba29d4158b540d9b7f0b75854eb49da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38089 } } }
I20260923 20:52:00.050729 26818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ba29d4158b540d9b7f0b75854eb49da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba29d4158b540d9b7f0b75854eb49da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38089 } } }
I20260923 20:52:00.051069 26818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:00.051070 26819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:00.053684 26824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:00.059463 26824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:00.070569 26824 catalog_manager.cc:1347] Generated new cluster ID: dc8712d2b2564b1d865001a9a5ab60e9
I20260923 20:52:00.070724 26824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:00.082839 26824 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:00.083415 26824 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:00.092648 26824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ba29d4158b540d9b7f0b75854eb49da: Generated new TSK 0
I20260923 20:52:00.093923 26824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:00.106812 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26740
W20260923 20:52:00.108570 26802 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.74:55679
W20260923 20:52:00.109509 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26740: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:00.109686 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26740
2026-09-23T20:52:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2079 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:00 dist-test-slave-dn29 krb5kdc[26838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:00 dist-test-slave-dn29 krb5kdc[26838](info): set up 2 sockets
Sep 23 20:52:00 dist-test-slave-dn29 krb5kdc[26838](info): commencing operation
krb5kdc: starting...
W20260923 20:52:01.268726 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:01 dist-test-slave-dn29 krb5kdc[26838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:52:01.560122 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:01.911851 26851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:01.912184 26851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:01.912351 26851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:01.920650 26851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:01.920783 26851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:01.920890 26851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:01.921003 26851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:01.921090 26851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:01.931392 26851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36921
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:01.932878 26851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:01.934288 26851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:01.946873 26859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:01.946933 26857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:01.947423 26856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:01.948096 26851 server_base.cc:1034] running on GCE node
I20260923 20:52:01.948948 26851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:01.950534 26851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:01.951858 26851 hybrid_clock.cc:648] HybridClock initialized: now 1790196721951762 us; error 49 us; skew 500 ppm
I20260923 20:52:01.952266 26851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:01 dist-test-slave-dn29 krb5kdc[26838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196721, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:01.955606 26851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:52:01.957393 26851 webserver.cc:460] Webserver started at http://127.0.0.1:34553/ using document root <none> and password file <none>
I20260923 20:52:01.958145 26851 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:01.958302 26851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:01.958622 26851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:01.961844 26851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d"
format_stamp: "Formatted at 2026-09-23 20:52:01 on dist-test-slave-dn29"
server_key: "ffc344c33301644cb9986acb4f70500e"
server_key_iv: "bd8cda6341a362fbbc063c0d816694cd"
server_key_version: "encryptionkey@0"
I20260923 20:52:01.962662 26851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d"
format_stamp: "Formatted at 2026-09-23 20:52:01 on dist-test-slave-dn29"
server_key: "ffc344c33301644cb9986acb4f70500e"
server_key_iv: "bd8cda6341a362fbbc063c0d816694cd"
server_key_version: "encryptionkey@0"
I20260923 20:52:01.968393 26851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260923 20:52:01.972568 26866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:01.974192 26851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:52:01.974449 26851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d"
format_stamp: "Formatted at 2026-09-23 20:52:01 on dist-test-slave-dn29"
server_key: "ffc344c33301644cb9986acb4f70500e"
server_key_iv: "bd8cda6341a362fbbc063c0d816694cd"
server_key_version: "encryptionkey@0"
I20260923 20:52:01.974819 26851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:01.995232 26851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:02.000607 26851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:02.000948 26851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:02.023391 26851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36921
I20260923 20:52:02.023440 26917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36921 every 8 connection(s)
I20260923 20:52:02.025089 26851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:02.029227 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26851
I20260923 20:52:02.029515 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:02.030002 5406 external_mini_cluster.cc:1428] Setting key d5e96ee9192b4e6693b240e1655a7a24
I20260923 20:52:02.030479 26918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:02.046598 26918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d: Bootstrap starting.
Sep 23 20:52:02 dist-test-slave-dn29 krb5kdc[26838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 20:52:02.050621 26918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:02.052162 26918 log.cc:824] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:02.055970 26918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d: No bootstrap required, opened a new log
I20260923 20:52:02.058827 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:02.031973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:47147 (local address 127.0.0.1:36921)
0923 20:52:02.032824 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:02.032836 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:02.032882 (+ 46us) server_negotiation.cc:407] Connection header received
0923 20:52:02.035443 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:02.035473 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:02.036146 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:02.036566 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:02.038662 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:02.040399 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:02.042383 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:02.043349 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:02.043519 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:02.048861 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:02.048872 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:02.048890 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:02.048944 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:02.053288 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:02.054558 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:02.054567 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:02.054576 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:02.054815 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:02.055346 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:02.055351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:02.055353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:02.056197 (+ 844us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:02.056555 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:02.057033 (+ 478us) server_negotiation.cc:299] Negotiation successful
0923 20:52:02.057375 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":398,"threads_started":1}
I20260923 20:52:02.063457 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:02.065747 26918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36921 } }
I20260923 20:52:02.066162 26918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:02.066231 26918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d72cd9769d545e3a4c9c7ce8def4b3d, State: Initialized, Role: FOLLOWER
I20260923 20:52:02.066893 26918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36921 } }
I20260923 20:52:02.067232 26918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:02.067397 26918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:02.067621 26918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:02.070518 26918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36921 } }
I20260923 20:52:02.071076 26918 leader_election.cc:302] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d72cd9769d545e3a4c9c7ce8def4b3d; no voters:
I20260923 20:52:02.071833 26918 leader_election.cc:288] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:02.071962 26928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:02.074735 26918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:02.078764 26928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [term 1 LEADER]: Becoming Leader. State: Replica: 6d72cd9769d545e3a4c9c7ce8def4b3d, State: Running, Role: LEADER
I20260923 20:52:02.079708 26928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36921 } }
I20260923 20:52:02.083441 26929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36921 } } }
I20260923 20:52:02.084560 26929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:02.084160 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d72cd9769d545e3a4c9c7ce8def4b3d. Latest consensus state: current_term: 1 leader_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d72cd9769d545e3a4c9c7ce8def4b3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36921 } } }
I20260923 20:52:02.085299 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:02.088011 26937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:02.090301 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:02.069162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:47343 (local address 127.0.0.1:36921)
0923 20:52:02.069404 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:02.069413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:02.069620 (+ 207us) server_negotiation.cc:407] Connection header received
0923 20:52:02.069876 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:02.069881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:02.069964 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:02.070085 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:02.071891 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:02.073327 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:02.074854 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:02.075556 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:02.075715 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:02.079062 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:02.079072 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:02.079077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:02.079128 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:02.083346 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:02.084625 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:02.084631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:02.084635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:02.084819 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:02.085757 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:02.085765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:02.085772 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:02.086558 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:02.089122 (+ 2564us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:02.089391 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 20:52:02.089685 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260923 20:52:02.098690 26937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:02.110965 26937 catalog_manager.cc:1347] Generated new cluster ID: d6688c341a254878a872dd8bafc205ac
I20260923 20:52:02.111217 26937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:02.125816 26937 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:02.127351 26937 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:02.141460 26937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d72cd9769d545e3a4c9c7ce8def4b3d: Generated new TSK 0
I20260923 20:52:02.142339 26937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:02.150533 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26851
I20260923 20:52:02.152743 26913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:02.255262 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26851
W20260923 20:52:02.273923 26836 connection.cc:535] client connection to 127.0.0.1:36921 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36921): Connection reset by peer (error 104)
2026-09-23T20:52:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2149 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:02 dist-test-slave-dn29 krb5kdc[26951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:02 dist-test-slave-dn29 krb5kdc[26951](info): set up 2 sockets
Sep 23 20:52:02 dist-test-slave-dn29 krb5kdc[26951](info): commencing operation
krb5kdc: starting...
W20260923 20:52:03.396890 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:03 dist-test-slave-dn29 krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:52:03.703577 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33687
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:04.053753 26964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:04.054157 26964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:04.054340 26964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:04.061808 26964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:04.061993 26964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:04.062137 26964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:04.062255 26964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:04.062346 26964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:04.072727 26964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33687
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:04.074179 26964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:04.075568 26964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:04.089105 26969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:04.089231 26970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:04.089200 26972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:04.090502 26964 server_base.cc:1034] running on GCE node
I20260923 20:52:04.091410 26964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:04.093058 26964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:04.094337 26964 hybrid_clock.cc:648] HybridClock initialized: now 1790196724094287 us; error 44 us; skew 500 ppm
I20260923 20:52:04.094779 26964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:04 dist-test-slave-dn29 krb5kdc[26951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:04.098517 26964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:52:04.100255 26964 webserver.cc:460] Webserver started at http://127.0.0.1:35179/ using document root <none> and password file <none>
I20260923 20:52:04.101030 26964 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:04.101153 26964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:04.101418 26964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:04.104940 26964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "f095cfd6389044649d0fe54d878b6901"
format_stamp: "Formatted at 2026-09-23 20:52:04 on dist-test-slave-dn29"
server_key: "11faee8818cd7066f1416112673d9274"
server_key_iv: "463076489f615dfa726377acfb57046e"
server_key_version: "encryptionkey@0"
I20260923 20:52:04.105835 26964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "f095cfd6389044649d0fe54d878b6901"
format_stamp: "Formatted at 2026-09-23 20:52:04 on dist-test-slave-dn29"
server_key: "11faee8818cd7066f1416112673d9274"
server_key_iv: "463076489f615dfa726377acfb57046e"
server_key_version: "encryptionkey@0"
I20260923 20:52:04.112059 26964 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260923 20:52:04.116072 26979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:04.117616 26964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:52:04.117779 26964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "f095cfd6389044649d0fe54d878b6901"
format_stamp: "Formatted at 2026-09-23 20:52:04 on dist-test-slave-dn29"
server_key: "11faee8818cd7066f1416112673d9274"
server_key_iv: "463076489f615dfa726377acfb57046e"
server_key_version: "encryptionkey@0"
I20260923 20:52:04.117959 26964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:04.140090 26964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:04.144927 26964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:04.145200 26964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:04.165252 26964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33687
I20260923 20:52:04.165307 27030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33687 every 8 connection(s)
I20260923 20:52:04.166712 26964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:04.170691 27031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:04.171888 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 26964
I20260923 20:52:04.172091 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:04.172631 5406 external_mini_cluster.cc:1428] Setting key 3bd0c4a232e75a4cdb6b4b384d17b85e
I20260923 20:52:04.184368 27031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901: Bootstrap starting.
I20260923 20:52:04.187264 27034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:04.174426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:44137 (local address 127.0.0.1:33687)
0923 20:52:04.175301 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:04.175319 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:04.175621 (+ 302us) server_negotiation.cc:407] Connection header received
0923 20:52:04.177824 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:04.177855 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:04.178313 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:04.178776 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:04.179349 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:04.180895 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:04.182345 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:04.183073 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:04.183283 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:04.184815 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:04.184824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:04.184843 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:04.185413 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:04.185491 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:04.185522 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:04.185733 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:04.186018 (+ 285us) server_negotiation.cc:299] Negotiation successful
0923 20:52:04.186210 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":362,"threads_started":1}
I20260923 20:52:04.187927 27031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:04.189188 27031 log.cc:824] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:04.192016 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:04.192682 27031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901: No bootstrap required, opened a new log
I20260923 20:52:04.201629 27031 raft_consensus.cc:348] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f095cfd6389044649d0fe54d878b6901" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33687 } }
I20260923 20:52:04.202191 27031 raft_consensus.cc:374] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:04.202316 27031 raft_consensus.cc:729] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f095cfd6389044649d0fe54d878b6901, State: Initialized, Role: FOLLOWER
I20260923 20:52:04.202965 27031 consensus_queue.cc:260] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f095cfd6389044649d0fe54d878b6901" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33687 } }
I20260923 20:52:04.203218 27031 raft_consensus.cc:388] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:04.203394 27031 raft_consensus.cc:482] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:04.203614 27031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:04.204380 27034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:04.197172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:38301 (local address 127.0.0.1:33687)
0923 20:52:04.197362 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:04.197368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:04.197710 (+ 342us) server_negotiation.cc:407] Connection header received
0923 20:52:04.197877 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:04.197884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:04.197970 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:04.198094 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:04.198654 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:04.199877 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:04.201338 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:04.201968 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:04.202122 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:04.203460 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:04.203467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:04.203473 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:04.203497 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:04.203546 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:04.203574 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:04.203710 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:04.203977 (+ 267us) server_negotiation.cc:299] Negotiation successful
0923 20:52:04.204043 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:52:04.206933 27031 raft_consensus.cc:504] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f095cfd6389044649d0fe54d878b6901" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33687 } }
I20260923 20:52:04.207433 27031 leader_election.cc:302] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f095cfd6389044649d0fe54d878b6901; no voters:
I20260923 20:52:04.208058 27031 leader_election.cc:288] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:04.208258 27041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:04.208981 27041 raft_consensus.cc:686] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [term 1 LEADER]: Becoming Leader. State: Replica: f095cfd6389044649d0fe54d878b6901, State: Running, Role: LEADER
I20260923 20:52:04.209583 27041 consensus_queue.cc:237] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f095cfd6389044649d0fe54d878b6901" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33687 } }
I20260923 20:52:04.210284 27031 sys_catalog.cc:564] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:04.213250 27042 sys_catalog.cc:455] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f095cfd6389044649d0fe54d878b6901" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f095cfd6389044649d0fe54d878b6901" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33687 } } }
I20260923 20:52:04.213512 27043 sys_catalog.cc:455] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f095cfd6389044649d0fe54d878b6901. Latest consensus state: current_term: 1 leader_uuid: "f095cfd6389044649d0fe54d878b6901" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f095cfd6389044649d0fe54d878b6901" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33687 } } }
I20260923 20:52:04.213882 27042 sys_catalog.cc:458] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:04.213945 27043 sys_catalog.cc:458] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:04.216234 27048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:04.221783 27048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:04.231408 27048 catalog_manager.cc:1347] Generated new cluster ID: d10a8b8eadbf4e3e8ef6e8206a146dc9
I20260923 20:52:04.231567 27048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:04.248175 27048 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:04.250155 27048 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:04.262458 27048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f095cfd6389044649d0fe54d878b6901: Generated new TSK 0
I20260923 20:52:04.263464 27048 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:04.283404 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26964
W20260923 20:52:04.285007 27026 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.74:44137
W20260923 20:52:04.285840 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 26964: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:04.285998 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 26964
W20260923 20:52:04.301302 26949 connection.cc:535] client connection to 127.0.0.1:33687 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33687): Connection reset by peer (error 104)
2026-09-23T20:52:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2026 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:04 dist-test-slave-dn29 krb5kdc[27062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:04 dist-test-slave-dn29 krb5kdc[27062](info): set up 2 sockets
Sep 23 20:52:04 dist-test-slave-dn29 krb5kdc[27062](info): commencing operation
krb5kdc: starting...
W20260923 20:52:05.413108 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:05 dist-test-slave-dn29 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 20:52:05.700850 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:06.046271 27075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:06.046671 27075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:06.046838 27075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:06.054410 27075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:06.054608 27075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:06.054723 27075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:06.054836 27075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:06.054916 27075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:06.065496 27075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:06.066939 27075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:06.068778 27075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:06.081351 27081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:06.081931 27080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:06.081411 27083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:06.082507 27075 server_base.cc:1034] running on GCE node
I20260923 20:52:06.083334 27075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:06.085053 27075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:06.086334 27075 hybrid_clock.cc:648] HybridClock initialized: now 1790196726086295 us; error 36 us; skew 500 ppm
I20260923 20:52:06.086745 27075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:06 dist-test-slave-dn29 krb5kdc[27062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:06.090138 27075 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 20:52:06.092029 27075 webserver.cc:460] Webserver started at http://127.0.0.1:44201/ using document root <none> and password file <none>
I20260923 20:52:06.092793 27075 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:06.092913 27075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:06.093194 27075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:06.096748 27075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "7544077aaa9b4c15b7a85d60f7e3d18f"
format_stamp: "Formatted at 2026-09-23 20:52:06 on dist-test-slave-dn29"
server_key: "cf538f2b5a8faf79a915ee4deb01518b"
server_key_iv: "5354e7867aeb56e49a35d1b0bd44e650"
server_key_version: "encryptionkey@0"
I20260923 20:52:06.097648 27075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "7544077aaa9b4c15b7a85d60f7e3d18f"
format_stamp: "Formatted at 2026-09-23 20:52:06 on dist-test-slave-dn29"
server_key: "cf538f2b5a8faf79a915ee4deb01518b"
server_key_iv: "5354e7867aeb56e49a35d1b0bd44e650"
server_key_version: "encryptionkey@0"
I20260923 20:52:06.103988 27075 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260923 20:52:06.108180 27090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:06.109946 27075 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260923 20:52:06.110172 27075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "7544077aaa9b4c15b7a85d60f7e3d18f"
format_stamp: "Formatted at 2026-09-23 20:52:06 on dist-test-slave-dn29"
server_key: "cf538f2b5a8faf79a915ee4deb01518b"
server_key_iv: "5354e7867aeb56e49a35d1b0bd44e650"
server_key_version: "encryptionkey@0"
I20260923 20:52:06.110388 27075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:06.116798 27075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:06.121484 27075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:06.121819 27075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:06.142804 27075 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34101
I20260923 20:52:06.142900 27141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34101 every 8 connection(s)
I20260923 20:52:06.144418 27075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:06.147933 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27075
I20260923 20:52:06.148132 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:06.148571 5406 external_mini_cluster.cc:1428] Setting key e579a50170a58553833fc467c12b7ba1
I20260923 20:52:06.148846 27142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:06.158775 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:06.150641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:35913 (local address 127.0.0.1:34101)
0923 20:52:06.151544 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:06.151575 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:06.151631 (+ 56us) server_negotiation.cc:407] Connection header received
0923 20:52:06.154309 (+ 2678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:06.154342 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:06.154862 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:06.155367 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:06.156582 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:06.156592 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:06.156612 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:06.156631 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:06.156724 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:06.156765 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:06.157023 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:06.157174 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 20:52:06.157405 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":355,"threads_started":1}
I20260923 20:52:06.162828 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:06.165268 27142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f: Bootstrap starting.
I20260923 20:52:06.169711 27142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:06.170452 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:06.167513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:58435 (local address 127.0.0.1:34101)
0923 20:52:06.167683 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:06.167688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:06.168042 (+ 354us) server_negotiation.cc:407] Connection header received
0923 20:52:06.168186 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:06.168192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:06.168259 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:06.168374 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:06.169411 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:06.169420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:06.169429 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:06.169454 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:06.169511 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:06.169538 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:06.169700 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:06.170023 (+ 323us) server_negotiation.cc:299] Negotiation successful
0923 20:52:06.170100 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 20:52:06.171159 27142 log.cc:824] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:06.174240 27142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f: No bootstrap required, opened a new log
I20260923 20:52:06.182669 27142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260923 20:52:06.183326 27142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:06.183493 27142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7544077aaa9b4c15b7a85d60f7e3d18f, State: Initialized, Role: FOLLOWER
I20260923 20:52:06.184324 27142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260923 20:52:06.184612 27142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:06.184810 27142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:06.185020 27142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:06.187496 27142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260923 20:52:06.188189 27142 leader_election.cc:302] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7544077aaa9b4c15b7a85d60f7e3d18f; no voters:
I20260923 20:52:06.189157 27142 leader_election.cc:288] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:06.189278 27152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:06.190784 27152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [term 1 LEADER]: Becoming Leader. State: Replica: 7544077aaa9b4c15b7a85d60f7e3d18f, State: Running, Role: LEADER
I20260923 20:52:06.191358 27152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260923 20:52:06.191732 27142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:06.194535 27153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } } }
I20260923 20:52:06.194486 27154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7544077aaa9b4c15b7a85d60f7e3d18f. Latest consensus state: current_term: 1 leader_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7544077aaa9b4c15b7a85d60f7e3d18f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } } }
I20260923 20:52:06.194991 27153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:06.194991 27154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:06.195959 27161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:06.201612 27161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:06.210232 27161 catalog_manager.cc:1347] Generated new cluster ID: 027f118a35ad4880afdb729980c2a75d
I20260923 20:52:06.210359 27161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:06.224647 27161 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:06.225171 27161 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:06.233219 27161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7544077aaa9b4c15b7a85d60f7e3d18f: Generated new TSK 0
I20260923 20:52:06.234138 27161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:06.247047 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27075
W20260923 20:52:06.248373 27137 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.74:35913
W20260923 20:52:06.249250 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27075: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:06.249406 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27075
2026-09-23T20:52:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1966 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:06 dist-test-slave-dn29 krb5kdc[27173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:06 dist-test-slave-dn29 krb5kdc[27173](info): set up 2 sockets
Sep 23 20:52:06 dist-test-slave-dn29 krb5kdc[27173](info): commencing operation
krb5kdc: starting...
W20260923 20:52:07.377134 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.008s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:07 dist-test-slave-dn29 krb5kdc[27173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:07.664547 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40893
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:08.009883 27186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:08.010210 27186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:08.010393 27186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:08.019197 27186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:08.019338 27186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:08.019431 27186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:08.019560 27186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:08.019661 27186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:08.030248 27186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:08.031737 27186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:08.033313 27186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:08.047106 27191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:08.047052 27194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:08.047106 27192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:08.048095 27186 server_base.cc:1034] running on GCE node
I20260923 20:52:08.048957 27186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:08.050525 27186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:08.051863 27186 hybrid_clock.cc:648] HybridClock initialized: now 1790196728051742 us; error 42 us; skew 500 ppm
I20260923 20:52:08.052295 27186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:08 dist-test-slave-dn29 krb5kdc[27173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196728, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:08.055639 27186 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:08.057399 27186 webserver.cc:460] Webserver started at http://127.5.71.190:33219/ using document root <none> and password file <none>
I20260923 20:52:08.058152 27186 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:08.058321 27186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:08.058650 27186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:08.061893 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "80ebc3e22d754a5f892ed9604097f334"
format_stamp: "Formatted at 2026-09-23 20:52:08 on dist-test-slave-dn29"
server_key: "e3b925250319043b078963aebb4d4555"
server_key_iv: "9ab8cfb04e834bace1675f3ac197e5ea"
server_key_version: "encryptionkey@0"
I20260923 20:52:08.062750 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "80ebc3e22d754a5f892ed9604097f334"
format_stamp: "Formatted at 2026-09-23 20:52:08 on dist-test-slave-dn29"
server_key: "e3b925250319043b078963aebb4d4555"
server_key_iv: "9ab8cfb04e834bace1675f3ac197e5ea"
server_key_version: "encryptionkey@0"
I20260923 20:52:08.068531 27186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:52:08.072412 27201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:08.073927 27186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:52:08.074139 27186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "80ebc3e22d754a5f892ed9604097f334"
format_stamp: "Formatted at 2026-09-23 20:52:08 on dist-test-slave-dn29"
server_key: "e3b925250319043b078963aebb4d4555"
server_key_iv: "9ab8cfb04e834bace1675f3ac197e5ea"
server_key_version: "encryptionkey@0"
I20260923 20:52:08.074373 27186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:08.096494 27186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:08.101243 27186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:08.101567 27186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:08.121963 27186 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:40893
I20260923 20:52:08.122066 27252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:40893 every 8 connection(s)
I20260923 20:52:08.123412 27186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:08.127614 27253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:08.132570 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27186
I20260923 20:52:08.132800 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:08.133404 5406 external_mini_cluster.cc:1428] Setting key c9930f0f29332e112da3498491676f7f
I20260923 20:52:08.142174 27253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334: Bootstrap starting.
I20260923 20:52:08.145671 27253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:08.146783 27253 log.cc:824] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:52:08 dist-test-slave-dn29 krb5kdc[27173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:52:08.149813 27253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334: No bootstrap required, opened a new log
I20260923 20:52:08.158427 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:08.135527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45793 (local address 127.5.71.190:40893)
0923 20:52:08.136463 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:08.136476 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:08.136519 (+ 43us) server_negotiation.cc:407] Connection header received
0923 20:52:08.138245 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:08.138271 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:08.138745 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:08.139185 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:08.140889 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:08.142214 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:08.143495 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:08.144282 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:08.144488 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:08.149499 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:08.149509 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:08.149527 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:08.149575 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:08.153335 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:08.154356 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:08.154364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:08.154381 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:08.154597 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:08.155184 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:08.155189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:08.155195 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:08.155920 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:08.156310 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:08.156912 (+ 602us) server_negotiation.cc:299] Negotiation successful
0923 20:52:08.157291 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":439,"threads_started":1}
I20260923 20:52:08.158345 27253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ebc3e22d754a5f892ed9604097f334" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40893 } }
I20260923 20:52:08.158910 27253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:08.159037 27253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ebc3e22d754a5f892ed9604097f334, State: Initialized, Role: FOLLOWER
I20260923 20:52:08.159745 27253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ebc3e22d754a5f892ed9604097f334" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40893 } }
I20260923 20:52:08.160076 27253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:08.160243 27253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:08.160431 27253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:08.163209 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:08.163331 27253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ebc3e22d754a5f892ed9604097f334" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40893 } }
I20260923 20:52:08.163959 27253 leader_election.cc:302] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80ebc3e22d754a5f892ed9604097f334; no voters:
I20260923 20:52:08.164577 27253 leader_election.cc:288] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:08.164793 27258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:08.166721 27253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:08.178314 27258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [term 1 LEADER]: Becoming Leader. State: Replica: 80ebc3e22d754a5f892ed9604097f334, State: Running, Role: LEADER
I20260923 20:52:08.179309 27258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ebc3e22d754a5f892ed9604097f334" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40893 } }
I20260923 20:52:08.182927 27260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80ebc3e22d754a5f892ed9604097f334. Latest consensus state: current_term: 1 leader_uuid: "80ebc3e22d754a5f892ed9604097f334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ebc3e22d754a5f892ed9604097f334" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40893 } } }
I20260923 20:52:08.183286 27260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:08.183951 27259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80ebc3e22d754a5f892ed9604097f334" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ebc3e22d754a5f892ed9604097f334" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40893 } } }
I20260923 20:52:08.184341 27259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334 [sys.catalog]: This master's current role is: LEADER
W20260923 20:52:08.188246 27277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:52:08.188450 27277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:52:08.188645 27278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:08.191406 27278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:08.199187 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:08.180651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55505 (local address 127.5.71.190:40893)
0923 20:52:08.180898 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:08.180905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:08.181353 (+ 448us) server_negotiation.cc:407] Connection header received
0923 20:52:08.181638 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:08.181642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:08.181711 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:08.181830 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:08.183959 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:08.185233 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:08.187051 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:08.187730 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:08.187877 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:08.191661 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:08.191668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:08.191673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:08.191715 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:08.195573 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:08.196695 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:08.196701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:08.196706 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:08.196854 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:08.197396 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:08.197401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:08.197405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:08.197871 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:08.198167 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:08.198605 (+ 438us) server_negotiation.cc:299] Negotiation successful
0923 20:52:08.198795 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 20:52:08.202463 27278 catalog_manager.cc:1347] Generated new cluster ID: f87deb851c93409da771a58a238bf63b
I20260923 20:52:08.202570 27278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:08.218959 27278 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:08.220917 27278 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:08.233827 27278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80ebc3e22d754a5f892ed9604097f334: Generated new TSK 0
I20260923 20:52:08.234942 27278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:08.246758 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27186
I20260923 20:52:08.248433 27248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:08.347343 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27186
W20260923 20:52:08.365224 27171 connection.cc:535] client connection to 127.5.71.190:40893 recv error: Network error: failed to read from TLS socket (remote: 127.5.71.190:40893): Connection reset by peer (error 104)
2026-09-23T20:52:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2096 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:08 dist-test-slave-dn29 krb5kdc[27285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:08 dist-test-slave-dn29 krb5kdc[27285](info): set up 2 sockets
Sep 23 20:52:08 dist-test-slave-dn29 krb5kdc[27285](info): commencing operation
krb5kdc: starting...
W20260923 20:52:09.540782 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:09 dist-test-slave-dn29 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:09.827775 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45651
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:10.175099 27298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:10.175509 27298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:10.175690 27298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:10.183089 27298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:10.183270 27298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:10.183380 27298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:10.183494 27298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:10.183585 27298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:10.193948 27298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:45651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:10.195518 27298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:10.196996 27298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:10.209441 27303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:10.209478 27304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:10.209453 27306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:10.210203 27298 server_base.cc:1034] running on GCE node
I20260923 20:52:10.211069 27298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:10.212615 27298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:10.213903 27298 hybrid_clock.cc:648] HybridClock initialized: now 1790196730213826 us; error 57 us; skew 500 ppm
I20260923 20:52:10.214288 27298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:10 dist-test-slave-dn29 krb5kdc[27285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196730, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:10.225175 27298 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:10.227061 27298 webserver.cc:460] Webserver started at http://127.5.71.190:43807/ using document root <none> and password file <none>
I20260923 20:52:10.227977 27298 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:10.228144 27298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:10.228499 27298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:10.232043 27298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "c192a4ef12724dc08b255a7b5d072bc2"
format_stamp: "Formatted at 2026-09-23 20:52:10 on dist-test-slave-dn29"
server_key: "08807a4afee9fe4ac9f3877e4aa95c4b"
server_key_iv: "a0a68cfadaa9c39a1e3dcc485834c29a"
server_key_version: "encryptionkey@0"
I20260923 20:52:10.232918 27298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "c192a4ef12724dc08b255a7b5d072bc2"
format_stamp: "Formatted at 2026-09-23 20:52:10 on dist-test-slave-dn29"
server_key: "08807a4afee9fe4ac9f3877e4aa95c4b"
server_key_iv: "a0a68cfadaa9c39a1e3dcc485834c29a"
server_key_version: "encryptionkey@0"
I20260923 20:52:10.239006 27298 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260923 20:52:10.243000 27313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:10.244612 27298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:10.244829 27298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "c192a4ef12724dc08b255a7b5d072bc2"
format_stamp: "Formatted at 2026-09-23 20:52:10 on dist-test-slave-dn29"
server_key: "08807a4afee9fe4ac9f3877e4aa95c4b"
server_key_iv: "a0a68cfadaa9c39a1e3dcc485834c29a"
server_key_version: "encryptionkey@0"
I20260923 20:52:10.245062 27298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:10.270500 27298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:10.275681 27298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:10.276043 27298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:10.297385 27298 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:45651
I20260923 20:52:10.297482 27364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:45651 every 8 connection(s)
I20260923 20:52:10.298887 27298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:10.302759 27365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:10.307046 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27298
I20260923 20:52:10.307225 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:10.307737 5406 external_mini_cluster.cc:1428] Setting key 22aa5060d4c3d460e3d9ad5460837661
I20260923 20:52:10.316107 27365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2: Bootstrap starting.
I20260923 20:52:10.319967 27365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:10.321282 27365 log.cc:824] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:10.324892 27365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2: No bootstrap required, opened a new log
I20260923 20:52:10.325193 27368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:10.309949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32995 (local address 127.5.71.190:45651)
0923 20:52:10.310815 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:10.310832 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:10.310886 (+ 54us) server_negotiation.cc:407] Connection header received
0923 20:52:10.314054 (+ 3168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:10.314093 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:10.314705 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:10.315187 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:10.315764 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:10.317774 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:10.319359 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:10.320513 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:10.320690 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:10.322296 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:10.322311 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:10.322335 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:10.323126 (+ 791us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:10.323206 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:10.323233 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:10.323359 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:10.323897 (+ 538us) server_negotiation.cc:299] Negotiation successful
0923 20:52:10.324107 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":365,"threads_started":1}
I20260923 20:52:10.329357 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:10.334249 27365 raft_consensus.cc:348] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c192a4ef12724dc08b255a7b5d072bc2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45651 } }
I20260923 20:52:10.334846 27365 raft_consensus.cc:374] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:10.334983 27365 raft_consensus.cc:729] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c192a4ef12724dc08b255a7b5d072bc2, State: Initialized, Role: FOLLOWER
I20260923 20:52:10.335706 27365 consensus_queue.cc:260] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c192a4ef12724dc08b255a7b5d072bc2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45651 } }
I20260923 20:52:10.336081 27365 raft_consensus.cc:388] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:10.336302 27365 raft_consensus.cc:482] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:10.336561 27365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:10.339090 27365 raft_consensus.cc:504] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c192a4ef12724dc08b255a7b5d072bc2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45651 } }
I20260923 20:52:10.339681 27365 leader_election.cc:302] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c192a4ef12724dc08b255a7b5d072bc2; no voters:
I20260923 20:52:10.340397 27365 leader_election.cc:288] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:10.340526 27375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:10.342015 27368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:10.334099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35399 (local address 127.5.71.190:45651)
0923 20:52:10.334334 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:10.334345 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:10.334949 (+ 604us) server_negotiation.cc:407] Connection header received
0923 20:52:10.335184 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:10.335190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:10.335273 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:10.335403 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:10.336016 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:10.337059 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:10.338497 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:10.339001 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:10.339157 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:10.340741 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:10.340751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:10.340759 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:10.340797 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:10.340869 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:10.340907 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:10.341093 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:10.341605 (+ 512us) server_negotiation.cc:299] Negotiation successful
0923 20:52:10.341664 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:52:10.342867 27375 raft_consensus.cc:686] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [term 1 LEADER]: Becoming Leader. State: Replica: c192a4ef12724dc08b255a7b5d072bc2, State: Running, Role: LEADER
I20260923 20:52:10.343175 27365 sys_catalog.cc:564] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:10.343715 27375 consensus_queue.cc:237] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c192a4ef12724dc08b255a7b5d072bc2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45651 } }
I20260923 20:52:10.346872 27377 sys_catalog.cc:455] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c192a4ef12724dc08b255a7b5d072bc2. Latest consensus state: current_term: 1 leader_uuid: "c192a4ef12724dc08b255a7b5d072bc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c192a4ef12724dc08b255a7b5d072bc2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45651 } } }
I20260923 20:52:10.347290 27377 sys_catalog.cc:458] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:10.348016 27376 sys_catalog.cc:455] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c192a4ef12724dc08b255a7b5d072bc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c192a4ef12724dc08b255a7b5d072bc2" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 45651 } } }
I20260923 20:52:10.348366 27376 sys_catalog.cc:458] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:10.348460 27382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:10.355851 27382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:10.365618 27382 catalog_manager.cc:1347] Generated new cluster ID: ddebcac56b184f75a606f84c1e7d7f0d
I20260923 20:52:10.365749 27382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:10.381332 27382 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:10.382814 27382 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:10.390380 27382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c192a4ef12724dc08b255a7b5d072bc2: Generated new TSK 0
I20260923 20:52:10.391330 27382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:10.407745 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27298
W20260923 20:52:10.409247 27360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32995
W20260923 20:52:10.410099 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27298: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:10.410230 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27298
W20260923 20:52:10.424453 27283 connection.cc:535] client connection to 127.5.71.190:45651 recv error: Network error: failed to read from TLS socket (remote: 127.5.71.190:45651): Connection reset by peer (error 104)
2026-09-23T20:52:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2058 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:10 dist-test-slave-dn29 krb5kdc[27396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:10 dist-test-slave-dn29 krb5kdc[27396](info): set up 2 sockets
Sep 23 20:52:10 dist-test-slave-dn29 krb5kdc[27396](info): commencing operation
krb5kdc: starting...
W20260923 20:52:11.564731 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:11 dist-test-slave-dn29 krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:11.839687 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:37835
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:43077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:37835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:12.198531 27409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:12.198875 27409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:12.199062 27409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:12.207921 27409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:12.208078 27409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:12.208171 27409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:12.208282 27409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:12.208353 27409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:12.218814 27409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:43077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:37835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:37835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:12.220261 27409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:12.221649 27409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:12.234330 27414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:12.234287 27415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:12.234239 27417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:12.235379 27409 server_base.cc:1034] running on GCE node
I20260923 20:52:12.236295 27409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:12.237767 27409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:12.238977 27409 hybrid_clock.cc:648] HybridClock initialized: now 1790196732238936 us; error 46 us; skew 500 ppm
I20260923 20:52:12.239298 27409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:12 dist-test-slave-dn29 krb5kdc[27396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196732, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:12.242898 27409 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:12.244649 27409 webserver.cc:460] Webserver started at http://127.5.71.190:40037/ using document root <none> and password file <none>
I20260923 20:52:12.245389 27409 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:12.245498 27409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:12.245913 27409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:12.249352 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "b45eb52691e548ab893af9b1e7d43fa8"
format_stamp: "Formatted at 2026-09-23 20:52:12 on dist-test-slave-dn29"
server_key: "cd974013c0fa5431d8341261ba9af63d"
server_key_iv: "4ea26fd3e4597f366497e6ae1cede68f"
server_key_version: "encryptionkey@0"
I20260923 20:52:12.250221 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "b45eb52691e548ab893af9b1e7d43fa8"
format_stamp: "Formatted at 2026-09-23 20:52:12 on dist-test-slave-dn29"
server_key: "cd974013c0fa5431d8341261ba9af63d"
server_key_iv: "4ea26fd3e4597f366497e6ae1cede68f"
server_key_version: "encryptionkey@0"
I20260923 20:52:12.256138 27409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:52:12.260188 27424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:12.261682 27409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260923 20:52:12.261911 27409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "b45eb52691e548ab893af9b1e7d43fa8"
format_stamp: "Formatted at 2026-09-23 20:52:12 on dist-test-slave-dn29"
server_key: "cd974013c0fa5431d8341261ba9af63d"
server_key_iv: "4ea26fd3e4597f366497e6ae1cede68f"
server_key_version: "encryptionkey@0"
I20260923 20:52:12.262108 27409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:12.268532 27409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:12.273591 27409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:12.273943 27409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:12.294077 27409 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:37835
I20260923 20:52:12.294121 27475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:37835 every 8 connection(s)
I20260923 20:52:12.295581 27409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:12.298244 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27409
I20260923 20:52:12.298434 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:12.298889 5406 external_mini_cluster.cc:1428] Setting key e7bd6a39ead07e1bf21e384b90b0dc17
I20260923 20:52:12.300547 27476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:12.308864 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:12.301061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42573 (local address 127.5.71.190:37835)
0923 20:52:12.301944 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:12.301961 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:12.302016 (+ 55us) server_negotiation.cc:407] Connection header received
0923 20:52:12.304366 (+ 2350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:12.304398 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:12.304902 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:12.305479 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:12.306598 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:12.306610 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:12.306637 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:12.306658 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:12.306752 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:12.306787 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:12.307022 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:12.307173 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 20:52:12.307413 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":358,"threads_started":1}
I20260923 20:52:12.313010 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:12.317301 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8: Bootstrap starting.
I20260923 20:52:12.321856 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:12.318955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53057 (local address 127.5.71.190:37835)
0923 20:52:12.319220 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:12.319236 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:12.319680 (+ 444us) server_negotiation.cc:407] Connection header received
0923 20:52:12.319920 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:12.319924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:12.319986 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:12.320087 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:12.321165 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:12.321169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:12.321172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:12.321187 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:12.321234 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:12.321250 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:12.321339 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:12.321491 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 20:52:12.321538 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:52:12.321914 27476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:12.323146 27476 log.cc:824] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:12.326288 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8: No bootstrap required, opened a new log
I20260923 20:52:12.334231 27476 raft_consensus.cc:348] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45eb52691e548ab893af9b1e7d43fa8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37835 } }
I20260923 20:52:12.334713 27476 raft_consensus.cc:374] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:12.334846 27476 raft_consensus.cc:729] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45eb52691e548ab893af9b1e7d43fa8, State: Initialized, Role: FOLLOWER
I20260923 20:52:12.335502 27476 consensus_queue.cc:260] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45eb52691e548ab893af9b1e7d43fa8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37835 } }
I20260923 20:52:12.335779 27476 raft_consensus.cc:388] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:12.336005 27476 raft_consensus.cc:482] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:12.336206 27476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:12.338631 27476 raft_consensus.cc:504] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45eb52691e548ab893af9b1e7d43fa8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37835 } }
I20260923 20:52:12.339227 27476 leader_election.cc:302] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b45eb52691e548ab893af9b1e7d43fa8; no voters:
I20260923 20:52:12.340075 27476 leader_election.cc:288] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:12.340170 27486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:12.341135 27486 raft_consensus.cc:686] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [term 1 LEADER]: Becoming Leader. State: Replica: b45eb52691e548ab893af9b1e7d43fa8, State: Running, Role: LEADER
I20260923 20:52:12.341823 27486 consensus_queue.cc:237] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45eb52691e548ab893af9b1e7d43fa8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37835 } }
I20260923 20:52:12.342787 27476 sys_catalog.cc:564] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:12.344910 27488 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b45eb52691e548ab893af9b1e7d43fa8. Latest consensus state: current_term: 1 leader_uuid: "b45eb52691e548ab893af9b1e7d43fa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45eb52691e548ab893af9b1e7d43fa8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37835 } } }
I20260923 20:52:12.345290 27488 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:12.345801 27487 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b45eb52691e548ab893af9b1e7d43fa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b45eb52691e548ab893af9b1e7d43fa8" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 37835 } } }
I20260923 20:52:12.346110 27487 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:12.346338 27492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:12.352461 27492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:12.362874 27492 catalog_manager.cc:1347] Generated new cluster ID: e149f3c5dba842cf9c0ac3d4aa476029
I20260923 20:52:12.363000 27492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:12.380678 27492 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:12.381210 27492 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:12.387611 27492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b45eb52691e548ab893af9b1e7d43fa8: Generated new TSK 0
I20260923 20:52:12.388640 27492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:12.393158 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27409
W20260923 20:52:12.394470 27471 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42573
W20260923 20:52:12.395241 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27409: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:12.395407 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27409
2026-09-23T20:52:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1991 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:12 dist-test-slave-dn29 krb5kdc[27507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:12 dist-test-slave-dn29 krb5kdc[27507](info): set up 2 sockets
Sep 23 20:52:12 dist-test-slave-dn29 krb5kdc[27507](info): commencing operation
krb5kdc: starting...
W20260923 20:52:13.560786 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:13 dist-test-slave-dn29 krb5kdc[27507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:13.856920 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:44211
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:44211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:14.199229 27520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:14.199543 27520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:14.199743 27520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:14.207396 27520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:14.207515 27520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:14.207701 27520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:14.207774 27520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:14.207885 27520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:14.218185 27520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:44211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:44211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:14.219566 27520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:14.220980 27520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:14.233781 27528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:14.233873 27526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:14.234381 27525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:14.235252 27520 server_base.cc:1034] running on GCE node
I20260923 20:52:14.236236 27520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:14.237735 27520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:14.238978 27520 hybrid_clock.cc:648] HybridClock initialized: now 1790196734238921 us; error 53 us; skew 500 ppm
I20260923 20:52:14.239348 27520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:14 dist-test-slave-dn29 krb5kdc[27507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196734, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:14.242719 27520 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:14.244362 27520 webserver.cc:460] Webserver started at http://127.5.71.190:34575/ using document root <none> and password file <none>
I20260923 20:52:14.245023 27520 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:14.245121 27520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:14.245393 27520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:14.248474 27520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "19ef92bfe1fc4294991cb3a4708da566"
format_stamp: "Formatted at 2026-09-23 20:52:14 on dist-test-slave-dn29"
server_key: "55c8c64ffcb44591e186207d49a9c432"
server_key_iv: "2ad94c6e44e24f23908ae3ec61b0ed21"
server_key_version: "encryptionkey@0"
I20260923 20:52:14.249228 27520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "19ef92bfe1fc4294991cb3a4708da566"
format_stamp: "Formatted at 2026-09-23 20:52:14 on dist-test-slave-dn29"
server_key: "55c8c64ffcb44591e186207d49a9c432"
server_key_iv: "2ad94c6e44e24f23908ae3ec61b0ed21"
server_key_version: "encryptionkey@0"
I20260923 20:52:14.254987 27520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260923 20:52:14.259212 27535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:14.260879 27520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260923 20:52:14.261087 27520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "19ef92bfe1fc4294991cb3a4708da566"
format_stamp: "Formatted at 2026-09-23 20:52:14 on dist-test-slave-dn29"
server_key: "55c8c64ffcb44591e186207d49a9c432"
server_key_iv: "2ad94c6e44e24f23908ae3ec61b0ed21"
server_key_version: "encryptionkey@0"
I20260923 20:52:14.261312 27520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:14.297343 27520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:14.302064 27520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:14.302376 27520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:14.323953 27520 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:44211
I20260923 20:52:14.324136 27586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:44211 every 8 connection(s)
I20260923 20:52:14.325455 27520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:14.329627 27587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:14.335011 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27520
I20260923 20:52:14.335251 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:14.335901 5406 external_mini_cluster.cc:1428] Setting key 7fe2ec65d69e6fbbcbac0a576383ee18
I20260923 20:52:14.342754 27587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566: Bootstrap starting.
I20260923 20:52:14.347088 27587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:14.348649 27587 log.cc:824] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:52:14 dist-test-slave-dn29 krb5kdc[27507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:52:14.352481 27587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566: No bootstrap required, opened a new log
I20260923 20:52:14.361335 27590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:14.337847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37123 (local address 127.5.71.190:44211)
0923 20:52:14.338698 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:14.338710 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:14.338752 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:52:14.340788 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:14.340814 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:14.341235 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:14.341573 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:14.343652 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:14.345262 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:14.347418 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:14.348346 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:14.348513 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:14.353640 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:14.353652 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:14.353670 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:14.353708 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:14.357046 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:14.357795 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:14.357805 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:14.357811 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:14.357983 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:14.358394 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:14.358399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:14.358404 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:14.358995 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:14.359341 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:14.359852 (+ 511us) server_negotiation.cc:299] Negotiation successful
0923 20:52:14.360296 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":405,"threads_started":1}
I20260923 20:52:14.361205 27587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ef92bfe1fc4294991cb3a4708da566" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44211 } }
I20260923 20:52:14.361698 27587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:14.361845 27587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19ef92bfe1fc4294991cb3a4708da566, State: Initialized, Role: FOLLOWER
I20260923 20:52:14.362445 27587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ef92bfe1fc4294991cb3a4708da566" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44211 } }
I20260923 20:52:14.362730 27587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:14.362893 27587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:14.363095 27587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:14.365314 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:14.366189 27587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ef92bfe1fc4294991cb3a4708da566" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44211 } }
I20260923 20:52:14.366791 27587 leader_election.cc:302] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19ef92bfe1fc4294991cb3a4708da566; no voters:
I20260923 20:52:14.367443 27587 leader_election.cc:288] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:14.367587 27592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:14.369679 27587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:14.375200 27592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [term 1 LEADER]: Becoming Leader. State: Replica: 19ef92bfe1fc4294991cb3a4708da566, State: Running, Role: LEADER
I20260923 20:52:14.376039 27592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ef92bfe1fc4294991cb3a4708da566" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44211 } }
I20260923 20:52:14.381467 27595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19ef92bfe1fc4294991cb3a4708da566. Latest consensus state: current_term: 1 leader_uuid: "19ef92bfe1fc4294991cb3a4708da566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ef92bfe1fc4294991cb3a4708da566" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44211 } } }
I20260923 20:52:14.381964 27595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:14.382210 27593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19ef92bfe1fc4294991cb3a4708da566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19ef92bfe1fc4294991cb3a4708da566" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 44211 } } }
I20260923 20:52:14.382529 27593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566 [sys.catalog]: This master's current role is: LEADER
W20260923 20:52:14.386648 27612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:52:14.386942 27612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:52:14.387131 27613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:14.389607 27613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:14.398475 27590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:14.378190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58883 (local address 127.5.71.190:44211)
0923 20:52:14.378441 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:14.378449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:14.379854 (+ 1405us) server_negotiation.cc:407] Connection header received
0923 20:52:14.380033 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:14.380037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:14.380107 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:14.380203 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:14.382877 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:14.383878 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:14.385752 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:14.386590 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:14.386760 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:14.390261 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:14.390271 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:14.390282 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:14.390337 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:14.394859 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:14.395859 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:14.395871 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:14.395878 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:14.396044 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:14.396541 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:14.396548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:14.396551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:14.397035 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:14.397314 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:14.397763 (+ 449us) server_negotiation.cc:299] Negotiation successful
0923 20:52:14.398022 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 20:52:14.399729 27613 catalog_manager.cc:1347] Generated new cluster ID: 520eb81693d64c0a9ac20eb2dc923074
I20260923 20:52:14.399865 27613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:14.424280 27613 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:14.425796 27613 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:14.434598 27613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19ef92bfe1fc4294991cb3a4708da566: Generated new TSK 0
I20260923 20:52:14.435490 27613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:14.443442 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27520
I20260923 20:52:14.445171 27582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:14.544073 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27520
2026-09-23T20:52:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:14 dist-test-slave-dn29 krb5kdc[27619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:14 dist-test-slave-dn29 krb5kdc[27619](info): set up 2 sockets
Sep 23 20:52:14 dist-test-slave-dn29 krb5kdc[27619](info): commencing operation
krb5kdc: starting...
W20260923 20:52:15.672715 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:15 dist-test-slave-dn29 krb5kdc[27619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:15.953475 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40859
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:40859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:16.307312 27632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:16.307641 27632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:16.307900 27632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:16.315550 27632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:16.315668 27632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:16.315762 27632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:16.315866 27632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:16.315948 27632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:16.326244 27632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:40859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:40859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:16.327595 27632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:16.328953 27632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:16.341528 27638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:16.341681 27637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:16.341701 27640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:16.342401 27632 server_base.cc:1034] running on GCE node
I20260923 20:52:16.343235 27632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:16.344854 27632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:16.346100 27632 hybrid_clock.cc:648] HybridClock initialized: now 1790196736346020 us; error 63 us; skew 500 ppm
I20260923 20:52:16.346568 27632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:16 dist-test-slave-dn29 krb5kdc[27619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196736, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:16.366918 27632 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:16.368871 27632 webserver.cc:460] Webserver started at http://127.5.71.190:40023/ using document root <none> and password file <none>
I20260923 20:52:16.369626 27632 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:16.369782 27632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:16.370100 27632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:16.373363 27632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "314577aaa80144128dc4bac72e4b3264"
format_stamp: "Formatted at 2026-09-23 20:52:16 on dist-test-slave-dn29"
server_key: "6a878c507fd6f84c44b5f08ad4f5653a"
server_key_iv: "b73e834ab46967467bffbe12b508d760"
server_key_version: "encryptionkey@0"
I20260923 20:52:16.374161 27632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "314577aaa80144128dc4bac72e4b3264"
format_stamp: "Formatted at 2026-09-23 20:52:16 on dist-test-slave-dn29"
server_key: "6a878c507fd6f84c44b5f08ad4f5653a"
server_key_iv: "b73e834ab46967467bffbe12b508d760"
server_key_version: "encryptionkey@0"
I20260923 20:52:16.379494 27632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:52:16.383591 27647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:16.385190 27632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:16.385396 27632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "314577aaa80144128dc4bac72e4b3264"
format_stamp: "Formatted at 2026-09-23 20:52:16 on dist-test-slave-dn29"
server_key: "6a878c507fd6f84c44b5f08ad4f5653a"
server_key_iv: "b73e834ab46967467bffbe12b508d760"
server_key_version: "encryptionkey@0"
I20260923 20:52:16.385627 27632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:16.398819 27632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:16.403422 27632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:16.403725 27632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:16.423492 27632 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:40859
I20260923 20:52:16.423523 27698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:40859 every 8 connection(s)
I20260923 20:52:16.425065 27632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:16.429100 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:16.432226 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27632
I20260923 20:52:16.432415 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:16.432971 5406 external_mini_cluster.cc:1428] Setting key 40ada67a55fcd2666e9fdaa0fedf4f10
I20260923 20:52:16.443111 27699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264: Bootstrap starting.
I20260923 20:52:16.447132 27699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:16.448468 27699 log.cc:824] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:16.450731 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:16.434954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50791 (local address 127.5.71.190:40859)
0923 20:52:16.435939 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:16.435958 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:16.436036 (+ 78us) server_negotiation.cc:407] Connection header received
0923 20:52:16.439149 (+ 3113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:16.439211 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:16.439933 (+ 722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:16.440524 (+ 591us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:16.441211 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:16.443697 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:16.445689 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:16.446429 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:16.446613 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:16.448011 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:16.448019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:16.448035 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:16.448628 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:16.448747 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:16.448785 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:16.448876 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:16.449184 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 20:52:16.449515 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":442,"threads_started":1}
I20260923 20:52:16.451890 27699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264: No bootstrap required, opened a new log
I20260923 20:52:16.454895 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:16.460810 27699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "314577aaa80144128dc4bac72e4b3264" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40859 } }
I20260923 20:52:16.461376 27699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:16.461485 27699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 314577aaa80144128dc4bac72e4b3264, State: Initialized, Role: FOLLOWER
I20260923 20:52:16.462116 27699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "314577aaa80144128dc4bac72e4b3264" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40859 } }
I20260923 20:52:16.462435 27699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:16.462622 27699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:16.462822 27699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:16.465364 27699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "314577aaa80144128dc4bac72e4b3264" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40859 } }
I20260923 20:52:16.465927 27699 leader_election.cc:302] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 314577aaa80144128dc4bac72e4b3264; no voters:
I20260923 20:52:16.466490 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:16.459659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34307 (local address 127.5.71.190:40859)
0923 20:52:16.459961 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:16.459970 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:16.460271 (+ 301us) server_negotiation.cc:407] Connection header received
0923 20:52:16.460475 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:16.460479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:16.460544 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:16.460629 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:16.461283 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:16.462292 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:16.463781 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:16.464338 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:16.464487 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:16.465640 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:16.465646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:16.465650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:16.465672 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:16.465726 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:16.465744 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:16.465848 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:16.465992 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 20:52:16.466085 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 20:52:16.466781 27699 leader_election.cc:288] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:16.466902 27709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:16.469233 27699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:16.472114 27709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [term 1 LEADER]: Becoming Leader. State: Replica: 314577aaa80144128dc4bac72e4b3264, State: Running, Role: LEADER
I20260923 20:52:16.472769 27709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "314577aaa80144128dc4bac72e4b3264" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40859 } }
I20260923 20:52:16.476209 27711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 314577aaa80144128dc4bac72e4b3264. Latest consensus state: current_term: 1 leader_uuid: "314577aaa80144128dc4bac72e4b3264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "314577aaa80144128dc4bac72e4b3264" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40859 } } }
I20260923 20:52:16.476673 27711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:16.476928 27710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "314577aaa80144128dc4bac72e4b3264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "314577aaa80144128dc4bac72e4b3264" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 40859 } } }
I20260923 20:52:16.477277 27710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264 [sys.catalog]: This master's current role is: LEADER
W20260923 20:52:16.481776 27723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:52:16.481931 27723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:52:16.482057 27725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:16.483759 27725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:16.493875 27725 catalog_manager.cc:1347] Generated new cluster ID: 0bf16e3479d645a4bcd8c1f0511a91cd
I20260923 20:52:16.494004 27725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:16.516808 27725 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:16.518792 27725 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:16.528568 27725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 314577aaa80144128dc4bac72e4b3264: Generated new TSK 0
I20260923 20:52:16.529542 27725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:16.545315 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27632
W20260923 20:52:16.546679 27694 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50791
W20260923 20:52:16.547432 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27632: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:16.547593 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27632
2026-09-23T20:52:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2001 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:16 dist-test-slave-dn29 krb5kdc[27730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:16 dist-test-slave-dn29 krb5kdc[27730](info): set up 2 sockets
Sep 23 20:52:16 dist-test-slave-dn29 krb5kdc[27730](info): commencing operation
krb5kdc: starting...
W20260923 20:52:17.681305 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:17 dist-test-slave-dn29 krb5kdc[27730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:17.969247 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:46205
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:32861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:46205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:18.323036 27743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:18.323365 27743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:18.323546 27743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:18.331092 27743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:18.331215 27743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:18.331334 27743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:18.331446 27743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:18.331540 27743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:18.341997 27743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:32861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:46205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:46205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:18.343425 27743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:18.344914 27743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:18.358227 27749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:18.358155 27751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:18.358251 27748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:18.361693 27743 server_base.cc:1034] running on GCE node
I20260923 20:52:18.362663 27743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:18.364400 27743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:18.365687 27743 hybrid_clock.cc:648] HybridClock initialized: now 1790196738365619 us; error 60 us; skew 500 ppm
I20260923 20:52:18.366112 27743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:18 dist-test-slave-dn29 krb5kdc[27730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:18.369819 27743 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:18.371556 27743 webserver.cc:460] Webserver started at http://127.5.71.190:37737/ using document root <none> and password file <none>
I20260923 20:52:18.372414 27743 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:18.372570 27743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:18.372884 27743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:18.376111 27743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "a60dd5c334c84d53a269a2e1b9065e98"
format_stamp: "Formatted at 2026-09-23 20:52:18 on dist-test-slave-dn29"
server_key: "17dc959c43f390286fc0cfedeb545f9b"
server_key_iv: "f229af746438f7bd71f5464c6e038e3c"
server_key_version: "encryptionkey@0"
I20260923 20:52:18.376945 27743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "a60dd5c334c84d53a269a2e1b9065e98"
format_stamp: "Formatted at 2026-09-23 20:52:18 on dist-test-slave-dn29"
server_key: "17dc959c43f390286fc0cfedeb545f9b"
server_key_iv: "f229af746438f7bd71f5464c6e038e3c"
server_key_version: "encryptionkey@0"
I20260923 20:52:18.382480 27743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260923 20:52:18.386257 27758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:18.388121 27743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:18.388350 27743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "a60dd5c334c84d53a269a2e1b9065e98"
format_stamp: "Formatted at 2026-09-23 20:52:18 on dist-test-slave-dn29"
server_key: "17dc959c43f390286fc0cfedeb545f9b"
server_key_iv: "f229af746438f7bd71f5464c6e038e3c"
server_key_version: "encryptionkey@0"
I20260923 20:52:18.388594 27743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:18.395535 27743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:18.400442 27743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:18.400774 27743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:18.422081 27743 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:46205
I20260923 20:52:18.422133 27809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:46205 every 8 connection(s)
I20260923 20:52:18.423614 27743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:18.427729 27810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:18.427893 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27743
I20260923 20:52:18.428072 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:18.428650 5406 external_mini_cluster.cc:1428] Setting key 3df6bfb669d9ba0245eae5c7c17e75b1
I20260923 20:52:18.436877 27813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:18.430525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38235 (local address 127.5.71.190:46205)
0923 20:52:18.431478 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:18.431492 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:18.431534 (+ 42us) server_negotiation.cc:407] Connection header received
0923 20:52:18.433154 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:18.433182 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:18.433547 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:18.433910 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:18.435065 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:18.435072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:18.435085 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:18.435107 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:18.435176 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:18.435196 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:18.435452 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:18.435562 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 20:52:18.435945 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":445,"threads_started":1}
I20260923 20:52:18.440867 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:18.442644 27810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98: Bootstrap starting.
I20260923 20:52:18.446512 27810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:18.447727 27810 log.cc:824] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:18.448421 27813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:18.445525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40449 (local address 127.5.71.190:46205)
0923 20:52:18.445732 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:18.445738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:18.446245 (+ 507us) server_negotiation.cc:407] Connection header received
0923 20:52:18.446457 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:18.446461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:18.446520 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:18.446632 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:18.447667 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:18.447673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:18.447678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 20:52:18.447697 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:18.447753 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 20:52:18.447770 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:18.447932 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:18.448049 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 20:52:18.448118 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 20:52:18.451303 27810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98: No bootstrap required, opened a new log
I20260923 20:52:18.459622 27810 raft_consensus.cc:348] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60dd5c334c84d53a269a2e1b9065e98" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46205 } }
I20260923 20:52:18.460233 27810 raft_consensus.cc:374] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 20:52:18.460351 27810 raft_consensus.cc:729] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a60dd5c334c84d53a269a2e1b9065e98, State: Initialized, Role: FOLLOWER
I20260923 20:52:18.461187 27810 consensus_queue.cc:260] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60dd5c334c84d53a269a2e1b9065e98" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46205 } }
I20260923 20:52:18.461515 27810 raft_consensus.cc:388] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 20:52:18.461671 27810 raft_consensus.cc:482] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 20:52:18.461813 27810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:18.464170 27810 raft_consensus.cc:504] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60dd5c334c84d53a269a2e1b9065e98" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46205 } }
I20260923 20:52:18.464721 27810 leader_election.cc:302] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a60dd5c334c84d53a269a2e1b9065e98; no voters:
I20260923 20:52:18.465342 27810 leader_election.cc:288] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 20:52:18.465592 27820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:18.466315 27820 raft_consensus.cc:686] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [term 1 LEADER]: Becoming Leader. State: Replica: a60dd5c334c84d53a269a2e1b9065e98, State: Running, Role: LEADER
I20260923 20:52:18.466881 27820 consensus_queue.cc:237] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60dd5c334c84d53a269a2e1b9065e98" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46205 } }
I20260923 20:52:18.467873 27810 sys_catalog.cc:564] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:18.470232 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a60dd5c334c84d53a269a2e1b9065e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60dd5c334c84d53a269a2e1b9065e98" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46205 } } }
I20260923 20:52:18.470130 27822 sys_catalog.cc:455] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a60dd5c334c84d53a269a2e1b9065e98. Latest consensus state: current_term: 1 leader_uuid: "a60dd5c334c84d53a269a2e1b9065e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a60dd5c334c84d53a269a2e1b9065e98" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46205 } } }
I20260923 20:52:18.470646 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:18.470705 27822 sys_catalog.cc:458] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:18.472882 27828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:18.478904 27828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:18.489627 27828 catalog_manager.cc:1347] Generated new cluster ID: 2763b14fa6d84fe7b9e6651aeb154e48
I20260923 20:52:18.489784 27828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:18.509660 27828 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:18.510150 27828 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:18.538748 27828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a60dd5c334c84d53a269a2e1b9065e98: Generated new TSK 0
I20260923 20:52:18.540040 27828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:18.552057 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27743
W20260923 20:52:18.553254 27805 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38235
W20260923 20:52:18.554113 5406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27743: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260923 20:52:18.554257 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27743
2026-09-23T20:52:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2006 ms)
[----------] 18 tests from AuthTokenIssuingTest (37250 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:18 dist-test-slave-dn29 krb5kdc[27841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:18 dist-test-slave-dn29 krb5kdc[27841](info): set up 2 sockets
Sep 23 20:52:18 dist-test-slave-dn29 krb5kdc[27841](info): commencing operation
krb5kdc: starting...
W20260923 20:52:19.692885 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:19 dist-test-slave-dn29 krb5kdc[27841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:19.984799 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:46675
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:46675,127.5.71.189:40001,127.5.71.188:42055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:20.343182 27854 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 20:52:20.343521 27854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:20.343587 27854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:20.343669 27854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:20.351760 27854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:20.351974 27854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:20.352053 27854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:20.352120 27854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:20.352192 27854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:20.362834 27854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:46675,127.5.71.189:40001,127.5.71.188:42055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:46675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:20.364315 27854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:20.365715 27854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:20.379307 27859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:20.379362 27860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:20.379334 27862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:20.380263 27854 server_base.cc:1034] running on GCE node
I20260923 20:52:20.381187 27854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:20.382730 27854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:20.383955 27854 hybrid_clock.cc:648] HybridClock initialized: now 1790196740383878 us; error 40 us; skew 500 ppm
I20260923 20:52:20.384361 27854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:20 dist-test-slave-dn29 krb5kdc[27841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:20.388320 27854 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:20.390129 27854 webserver.cc:460] Webserver started at http://127.5.71.190:35377/ using document root <none> and password file <none>
I20260923 20:52:20.390975 27854 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:20.391121 27854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:20.391498 27854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:20.395009 27854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "e3b00a0bf4794439bee90c7de413ffc9"
format_stamp: "Formatted at 2026-09-23 20:52:20 on dist-test-slave-dn29"
server_key: "cb130106976a9ec159dfd76723efaf76"
server_key_iv: "e6b5b717c0d3c9d6e2038a0d6fb9fe30"
server_key_version: "encryptionkey@0"
I20260923 20:52:20.395991 27854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "e3b00a0bf4794439bee90c7de413ffc9"
format_stamp: "Formatted at 2026-09-23 20:52:20 on dist-test-slave-dn29"
server_key: "cb130106976a9ec159dfd76723efaf76"
server_key_iv: "e6b5b717c0d3c9d6e2038a0d6fb9fe30"
server_key_version: "encryptionkey@0"
I20260923 20:52:20.401837 27854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260923 20:52:20.405719 27869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:20.407284 27854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 20:52:20.407586 27854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "e3b00a0bf4794439bee90c7de413ffc9"
format_stamp: "Formatted at 2026-09-23 20:52:20 on dist-test-slave-dn29"
server_key: "cb130106976a9ec159dfd76723efaf76"
server_key_iv: "e6b5b717c0d3c9d6e2038a0d6fb9fe30"
server_key_version: "encryptionkey@0"
I20260923 20:52:20.407891 27854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:20.431694 27854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:20.436553 27854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:20.436877 27854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:20.457347 27854 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:46675
I20260923 20:52:20.457367 27920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:46675 every 8 connection(s)
I20260923 20:52:20.459121 27854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:20.463379 27921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:20.463761 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27854
I20260923 20:52:20.464003 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:20.464869 5406 external_mini_cluster.cc:1428] Setting key e1392b2cbd40b4eb73f5fd4d09c5855c
I20260923 20:52:20.474478 27921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:20 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.189@KRBTEST.COM" created.
I20260923 20:52:20.503626 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.478842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:46675 (local address 127.0.0.1:41666)
0923 20:52:20.479817 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.479881 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:20.482171 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:20.483332 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:20.483338 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:20.484143 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:20.484506 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:20.484526 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:20.487264 (+ 2738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:20.487272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:20.488741 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:20.488757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:20.490451 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:20.490457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:20.490779 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:20.491683 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:20.491712 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:20.494482 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:20.498724 (+ 4242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:20.498737 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:20.498749 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:20.499217 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:20.499895 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:20.499899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:20.499902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:20.500104 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:20.501298 (+ 1194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:20.501306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:20.501536 (+ 230us) client_negotiation.cc:769] Sending connection context
0923 20:52:20.501836 (+ 300us) client_negotiation.cc:240] Negotiation successful
0923 20:52:20.502346 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":59,"thread_start_us":414,"threads_started":1}
I20260923 20:52:20.503676 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.480349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.5.71.190:46675)
0923 20:52:20.481194 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:20.481210 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:20.481281 (+ 71us) server_negotiation.cc:407] Connection header received
0923 20:52:20.482640 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:20.482651 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:20.483018 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:20.483278 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:20.484764 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:20.486985 (+ 2221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:20.488927 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:20.490145 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:20.490385 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:20.494829 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:20.494847 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:20.494872 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:20.494929 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:20.498523 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:20.499414 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:20.499428 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:20.499435 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:20.499655 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:20.500263 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:20.500271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:20.500277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:20.501089 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:20.501328 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:20.502029 (+ 701us) server_negotiation.cc:299] Negotiation successful
0923 20:52:20.502546 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":246,"server-negotiator.queue_time_us":570,"thread_start_us":393,"threads_started":1}
I20260923 20:52:20.508160 27921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:20.509795 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.509090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58768)
0923 20:52:20.509448 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.509608 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
W20260923 20:52:20.510259 27870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
I20260923 20:52:20.512763 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.512113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58778)
0923 20:52:20.512402 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.512509 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260923 20:52:20.513041 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:40001: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab.
Entry for principal kudu/127.5.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab.
I20260923 20:52:20.540513 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:40001
--webserver_interface=127.5.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab
--principal=kudu/127.5.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:46675,127.5.71.189:40001,127.5.71.188:42055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:52:20.562479 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } attempt: 1
I20260923 20:52:20.563998 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.563357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58792)
0923 20:52:20.563627 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.563758 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260923 20:52:20.565255 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.564785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58804)
0923 20:52:20.564969 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.565073 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260923 20:52:20.565522 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:40001: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
I20260923 20:52:20.633862 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } attempt: 2
I20260923 20:52:20.635396 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.634752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58816)
0923 20:52:20.635065 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.635203 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260923 20:52:20.636734 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.636180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58828)
0923 20:52:20.636381 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.636517 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260923 20:52:20.637014 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:40001: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
I20260923 20:52:20.728319 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } attempt: 3
I20260923 20:52:20.729926 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.729259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58834)
0923 20:52:20.729539 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.729687 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260923 20:52:20.731458 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.730873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58848)
0923 20:52:20.731091 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.731213 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260923 20:52:20.731825 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:40001: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
I20260923 20:52:20.875168 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } attempt: 4
I20260923 20:52:20.876755 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.876013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58856)
0923 20:52:20.876402 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.876541 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260923 20:52:20.877971 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:20.877548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58870)
0923 20:52:20.877698 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:20.877847 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260923 20:52:20.878285 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:40001: Network error: Client connection negotiation failed: client connection to 127.5.71.189:40001: connect: Connection refused (error 111)
W20260923 20:52:20.904428 27928 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 20:52:20.904839 27928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:20.904989 27928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:20.905171 27928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:20.913693 27928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:20.913813 27928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:20.913913 27928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:20.913995 27928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:20.914098 27928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:20.925024 27928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:46675,127.5.71.189:40001,127.5.71.188:42055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab
--principal=kudu/127.5.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:40001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:20.926476 27928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:20.927994 27928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:20.941263 27936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:20.941200 27934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:20.942104 27933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:20.942102 27928 server_base.cc:1034] running on GCE node
I20260923 20:52:20.943163 27928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:20.945032 27928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:20.946347 27928 hybrid_clock.cc:648] HybridClock initialized: now 1790196740946282 us; error 40 us; skew 500 ppm
I20260923 20:52:20.946825 27928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:20 dist-test-slave-dn29 krb5kdc[27841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:20.950562 27928 init.cc:377] Logged in from keytab as kudu/127.5.71.189@KRBTEST.COM (short username kudu)
I20260923 20:52:20.952376 27928 webserver.cc:460] Webserver started at http://127.5.71.189:46647/ using document root <none> and password file <none>
I20260923 20:52:20.953122 27928 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:20.953262 27928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:20.953598 27928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:20.956898 27928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data/instance:
uuid: "9daf674a6a834cd194d0ae738f03e479"
format_stamp: "Formatted at 2026-09-23 20:52:20 on dist-test-slave-dn29"
server_key: "a4f2241f5ccbe1362150246c1ffac607"
server_key_iv: "6ab41fe2ab917e2dafd2f38f4f85c660"
server_key_version: "encryptionkey@0"
I20260923 20:52:20.957731 27928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal/instance:
uuid: "9daf674a6a834cd194d0ae738f03e479"
format_stamp: "Formatted at 2026-09-23 20:52:20 on dist-test-slave-dn29"
server_key: "a4f2241f5ccbe1362150246c1ffac607"
server_key_iv: "6ab41fe2ab917e2dafd2f38f4f85c660"
server_key_version: "encryptionkey@0"
I20260923 20:52:20.963502 27928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260923 20:52:20.967366 27943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:20.969135 27928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:20.969339 27928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal
uuid: "9daf674a6a834cd194d0ae738f03e479"
format_stamp: "Formatted at 2026-09-23 20:52:20 on dist-test-slave-dn29"
server_key: "a4f2241f5ccbe1362150246c1ffac607"
server_key_iv: "6ab41fe2ab917e2dafd2f38f4f85c660"
server_key_version: "encryptionkey@0"
I20260923 20:52:20.969583 27928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:21.014627 27928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:21.019326 27928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:21.019717 27928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:21.040508 27928 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.189:40001
I20260923 20:52:21.040556 27994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.189:40001 every 8 connection(s)
I20260923 20:52:21.041935 27928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
I20260923 20:52:21.045753 27995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:21.050493 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 27928
I20260923 20:52:21.050778 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-1/wal/instance
I20260923 20:52:21.051321 5406 external_mini_cluster.cc:1428] Setting key 8ed80e3576e1cb1c0b7a0e4635d0ec2d
I20260923 20:52:21.052944 27995 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.188@KRBTEST.COM; defaulting to no policy
I20260923 20:52:21.077986 27997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.055763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.5.71.190:46675)
0923 20:52:21.056347 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.056352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.057537 (+ 1185us) server_negotiation.cc:407] Connection header received
0923 20:52:21.059497 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.059503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.059595 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.059829 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.062065 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.063010 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.065863 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.066375 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.066503 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.071200 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.071207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.071210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.071280 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.074592 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.075428 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.075434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.075438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.075574 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.076111 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.076116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.076119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.076424 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.076600 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.077358 (+ 758us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.077597 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":293,"threads_started":1}
I20260923 20:52:21.079088 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.056492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:46675 (local address 127.0.0.1:41674)
0923 20:52:21.057355 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.057395 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.059196 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.060015 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.060034 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.060880 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.061739 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.061766 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.063219 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.063227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.065497 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.065518 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.066648 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.066657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.067009 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.067913 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.067949 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.070907 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.074753 (+ 3846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.074764 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.074784 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.075284 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.075701 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.075706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.075710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.075983 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.076563 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.076573 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.076970 (+ 397us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.077261 (+ 291us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.077856 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":374,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.188@KRBTEST.COM" created.
I20260923 20:52:21.081274 27995 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:21.100508 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.082256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58884)
0923 20:52:21.082474 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.082491 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.082671 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.084005 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.084013 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.084538 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.084752 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.084765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.086273 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.086281 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.087824 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.087841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.089059 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.089072 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.089325 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.090007 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.090025 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.092669 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.097452 (+ 4783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.097457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.097460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.097873 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.098406 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.098411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.098413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.098659 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.099484 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.099488 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.099725 (+ 237us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.099953 (+ 228us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.100206 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260923 20:52:21.100997 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.082651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.5.71.189:40001)
0923 20:52:21.083401 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.083408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.083439 (+ 31us) server_negotiation.cc:407] Connection header received
0923 20:52:21.083491 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.083498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.083733 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.083990 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.084967 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.086106 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.087985 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.088897 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.089001 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.092933 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.092952 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.092977 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.093022 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.097285 (+ 4263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.098029 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.098042 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.098048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.098217 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.098790 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.098797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.098803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.099325 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.099564 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.100138 (+ 574us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.100462 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":416,"threads_started":1}
I20260923 20:52:21.104367 27995 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:21.105654 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.105176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42698)
0923 20:52:21.105367 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.105494 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260923 20:52:21.105947 27945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.107575 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.107224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42712)
0923 20:52:21.107402 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.107469 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260923 20:52:21.107785 27995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab.
Entry for principal kudu/127.5.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab.
I20260923 20:52:21.120298 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:42055
--webserver_interface=127.5.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab
--principal=kudu/127.5.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:46675,127.5.71.189:40001,127.5.71.188:42055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:52:21.157168 27995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 1
I20260923 20:52:21.158339 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.157866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42726)
0923 20:52:21.158069 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.158188 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260923 20:52:21.159384 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.159048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42730)
0923 20:52:21.159191 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.159282 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 20:52:21.159591 27995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.177552 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } attempt: 5
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:21.193219 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.178524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.5.71.189:40001)
0923 20:52:21.178733 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.178739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.178764 (+ 25us) server_negotiation.cc:407] Connection header received
0923 20:52:21.178923 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.178929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.179027 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.179134 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.180001 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.181086 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.182282 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.182966 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.183095 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.186496 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.186503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.186508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.186549 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.190234 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.190948 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.190954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.190957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.191111 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.191591 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.191596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.191599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.192054 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.192249 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.192642 (+ 393us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.192868 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:52:21.193238 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.178321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58890)
0923 20:52:21.178548 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.178571 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.178790 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.179177 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.179180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.179570 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.179852 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.179864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.181249 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.181253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.182134 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.182146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.183110 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.183114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.183272 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.183961 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.183980 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.186269 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.190390 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.190394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.190397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.190820 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.191293 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.191297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.191300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.191440 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.192220 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.192225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.192382 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.192531 (+ 149us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.192868 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260923 20:52:21.194523 27921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:21.195583 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.195185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42734)
0923 20:52:21.195362 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.195445 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260923 20:52:21.196529 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.196188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42736)
0923 20:52:21.196357 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.196426 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260923 20:52:21.196730 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.227864 27995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 2
I20260923 20:52:21.229136 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.228658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42746)
0923 20:52:21.228880 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.228975 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260923 20:52:21.230435 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.229999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42752)
0923 20:52:21.230187 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.230285 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260923 20:52:21.230680 27995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.247951 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 1
I20260923 20:52:21.249118 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.248644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42758)
0923 20:52:21.248848 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.248955 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260923 20:52:21.250092 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.249767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42770)
0923 20:52:21.249911 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.249988 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260923 20:52:21.250327 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.318564 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 2
I20260923 20:52:21.319732 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.319238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42772)
0923 20:52:21.319477 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.319573 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260923 20:52:21.320776 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.320425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42780)
0923 20:52:21.320607 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.320671 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 20:52:21.320990 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.321929 27995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 3
I20260923 20:52:21.323141 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.322542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42790)
0923 20:52:21.322804 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.322927 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260923 20:52:21.324460 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.323896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42796)
0923 20:52:21.324161 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.324275 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260923 20:52:21.324846 27995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
I20260923 20:52:21.427229 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 3
I20260923 20:52:21.428406 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.427950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42808)
0923 20:52:21.428141 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.428240 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 20:52:21.429658 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.429126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42812)
0923 20:52:21.429373 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.429484 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260923 20:52:21.429961 27921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
W20260923 20:52:21.466840 28003 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 20:52:21.467203 28003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:21.467336 28003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:21.467464 28003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 20:52:21.468076 27995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 4
I20260923 20:52:21.469282 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.468780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42820)
0923 20:52:21.468995 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.469098 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260923 20:52:21.470448 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.470062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42834)
0923 20:52:21.470231 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.470314 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260923 20:52:21.470781 27995 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:42055: Network error: Client connection negotiation failed: client connection to 127.5.71.188:42055: connect: Connection refused (error 111)
W20260923 20:52:21.476233 28003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:21.476363 28003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:21.476483 28003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:21.476570 28003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:21.476678 28003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:21.487037 28003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:46675,127.5.71.189:40001,127.5.71.188:42055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab
--principal=kudu/127.5.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:42055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:21.488538 28003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:21.489938 28003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:21.503001 28011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:21.502983 28008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:21.503670 28009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:21.503681 28003 server_base.cc:1034] running on GCE node
I20260923 20:52:21.504767 28003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:21.506496 28003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:21.507954 28003 hybrid_clock.cc:648] HybridClock initialized: now 1790196741507877 us; error 56 us; skew 500 ppm
I20260923 20:52:21.508356 28003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:21.523522 28003 init.cc:377] Logged in from keytab as kudu/127.5.71.188@KRBTEST.COM (short username kudu)
I20260923 20:52:21.525367 28003 webserver.cc:460] Webserver started at http://127.5.71.188:33231/ using document root <none> and password file <none>
I20260923 20:52:21.526324 28003 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:21.526474 28003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:21.526850 28003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:21.530246 28003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data/instance:
uuid: "490c519afbd44fb6ab4184a36626e5e8"
format_stamp: "Formatted at 2026-09-23 20:52:21 on dist-test-slave-dn29"
server_key: "5df60248912daf615386f9302b83d1d6"
server_key_iv: "ddcf7cf390b64fe8ec8a617fd5879578"
server_key_version: "encryptionkey@0"
I20260923 20:52:21.531224 28003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal/instance:
uuid: "490c519afbd44fb6ab4184a36626e5e8"
format_stamp: "Formatted at 2026-09-23 20:52:21 on dist-test-slave-dn29"
server_key: "5df60248912daf615386f9302b83d1d6"
server_key_iv: "ddcf7cf390b64fe8ec8a617fd5879578"
server_key_version: "encryptionkey@0"
I20260923 20:52:21.537211 28003 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260923 20:52:21.541047 28018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:21.542641 28003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260923 20:52:21.542862 28003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal
uuid: "490c519afbd44fb6ab4184a36626e5e8"
format_stamp: "Formatted at 2026-09-23 20:52:21 on dist-test-slave-dn29"
server_key: "5df60248912daf615386f9302b83d1d6"
server_key_iv: "ddcf7cf390b64fe8ec8a617fd5879578"
server_key_version: "encryptionkey@0"
I20260923 20:52:21.543115 28003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:21.561635 28003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:21.566097 28003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:21.566383 28003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:21.586715 28003 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.188:42055
I20260923 20:52:21.586756 28069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.188:42055 every 8 connection(s)
I20260923 20:52:21.588231 28003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
I20260923 20:52:21.592516 28070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:21.597615 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 28003
I20260923 20:52:21.597780 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790196359367181-5406-0/minicluster-data/master-2/wal/instance
I20260923 20:52:21.598318 5406 external_mini_cluster.cc:1428] Setting key 77dc2862bb07854b79acd31a01a9fbfc
I20260923 20:52:21.600561 28070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:21.607230 27921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 4
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:52:21.626257 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.604255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41682 (local address 127.5.71.190:46675)
0923 20:52:21.605768 (+ 1513us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.605773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.605789 (+ 16us) server_negotiation.cc:407] Connection header received
0923 20:52:21.605841 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.605844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.605915 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.606007 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.607922 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.609093 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.611437 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.612255 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.612388 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.616825 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.616831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.616835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.616875 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.621171 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.622250 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.622257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.622261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.622401 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.622881 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.622889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.622893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.623391 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.623557 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.625543 (+ 1986us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.625818 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1410,"thread_start_us":1238,"threads_started":1}
I20260923 20:52:21.626358 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.600448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.5.71.188:42055)
0923 20:52:21.601354 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.601371 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.601427 (+ 56us) server_negotiation.cc:407] Connection header received
0923 20:52:21.604080 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.604123 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.604608 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.605091 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.607443 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.609638 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.611527 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.612206 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.612403 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.617244 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.617255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.617274 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.617322 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.621804 (+ 4482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.622568 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.622579 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.622584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.622801 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.623319 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.623326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.623331 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.624115 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.624454 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.624811 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.625180 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":608,"spinlock_wait_cycles":18176,"thread_start_us":345,"threads_started":1}
I20260923 20:52:21.627139 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.604225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:46675 (local address 127.0.0.1:41682)
0923 20:52:21.605109 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.605164 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.605305 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.606127 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.606137 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.607020 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.607687 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.607714 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.609316 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.609322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.611243 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.611273 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.612719 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.612726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.612928 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.613542 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.613566 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.616542 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.621357 (+ 4815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.621378 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.621398 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.622107 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.622522 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.622528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.622532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.622752 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.623528 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.623539 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.624079 (+ 540us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.624435 (+ 356us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.625041 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":28,"spinlock_wait_cycles":23808,"thread_start_us":414,"threads_started":1}
I20260923 20:52:21.627877 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.607939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42866)
0923 20:52:21.608196 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.608214 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.608323 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.609274 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.609281 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.609803 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.609970 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.609979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.611385 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.611395 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.613118 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.613136 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.616085 (+ 2949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.616092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.616330 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.617262 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.617280 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.620003 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.624187 (+ 4184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.624193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.624197 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.624728 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.625480 (+ 752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.625486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.625489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.625637 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.626675 (+ 1038us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.626680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.626957 (+ 277us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.627132 (+ 175us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.627504 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260923 20:52:21.628942 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.608167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.5.71.188:42055)
0923 20:52:21.608946 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.608952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.608974 (+ 22us) server_negotiation.cc:407] Connection header received
0923 20:52:21.609019 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.609030 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.609093 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.609202 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.610146 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.611222 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.615142 (+ 3920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.615903 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.616042 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.620242 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.620247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.620250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.620278 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.624039 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.624879 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.624885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.624888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.625065 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.625796 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.625800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.625804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.626205 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.626683 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.628331 (+ 1648us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.628517 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":659,"thread_start_us":364,"threads_started":1}
I20260923 20:52:21.628917 28070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:21.636474 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: Bootstrap starting.
I20260923 20:52:21.641396 27921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: Neither blocks nor log segments found. Creating new log.
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:52:21.642982 27921 log.cc:824] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:21.643241 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.629716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:40001 (local address 127.0.0.1:58892)
0923 20:52:21.629892 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.629920 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.630042 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.630437 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.630442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.630946 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.631177 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.631190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.632305 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.632312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.633215 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.633236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.633940 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.633945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.634122 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.634776 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.634794 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.637270 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.640535 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.640540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.640544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.640951 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.641350 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.641355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.641359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.641531 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.642163 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.642169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.642357 (+ 188us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.642521 (+ 164us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.642828 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 20:52:21.643349 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.629822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.5.71.189:40001)
0923 20:52:21.630062 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.630068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.630106 (+ 38us) server_negotiation.cc:407] Connection header received
0923 20:52:21.630176 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.630181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.630261 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.630385 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.631373 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.632163 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.633369 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.633791 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.633930 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.637479 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.637484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.637487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.637522 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.640393 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.641077 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.641082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.641086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.641234 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.641729 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.641733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.641735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.642024 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.642201 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.642671 (+ 470us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.642940 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:52:21.644449 28070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:21.646665 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: No bootstrap required, opened a new log
I20260923 20:52:21.653056 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.630695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.5.71.190:46675)
0923 20:52:21.632003 (+ 1308us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.632009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.632029 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:52:21.632078 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.632100 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.632196 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.632308 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.634196 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.635342 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.637299 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.638044 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.638246 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.644845 (+ 6599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.644852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.644859 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.644897 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.648925 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.649860 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.649866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.649869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.650004 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.650428 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.650434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.650441 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.650808 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.651098 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.652384 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.652639 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1176}
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:52:21.658231 27921 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.658694 27921 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b00a0bf4794439bee90c7de413ffc9, State: Initialized, Role: FOLLOWER
I20260923 20:52:21.659288 27921 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.660478 28078 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.661000 28078 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.661152 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.645369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42868)
0923 20:52:21.645515 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.645539 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.645639 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.646015 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.646021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.646546 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.646744 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.646755 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.648287 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.648293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.649714 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.649731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.650827 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.650834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.651036 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.651722 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.651746 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.654682 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.658392 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.658400 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.658404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.658869 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.659231 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.659237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.659243 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.659380 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.660111 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.660116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.660292 (+ 176us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.660447 (+ 155us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.660765 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 20:52:21.662006 27921 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:21.662181 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.645343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.5.71.188:42055)
0923 20:52:21.645573 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.645579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.645599 (+ 20us) server_negotiation.cc:407] Connection header received
0923 20:52:21.645730 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.645738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.645814 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.645965 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.646878 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.648133 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.649872 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.650658 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.650803 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.654889 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.654895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.654899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.654934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.658258 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.658992 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.658999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.659002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.659125 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.659481 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.659489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.659492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.659969 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.661551 (+ 1582us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.661654 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.661855 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:21.670099 28070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: Bootstrap starting.
I20260923 20:52:21.675118 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.654941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58900 (local address 127.5.71.189:40001)
0923 20:52:21.655137 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.655143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.655213 (+ 70us) server_negotiation.cc:407] Connection header received
0923 20:52:21.655383 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.655389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.655473 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.655992 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.657632 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.658894 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.660958 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.661704 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.661846 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.666870 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.666876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.666879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.666916 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.670528 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.671611 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.671619 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.671624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.671871 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.672389 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.672395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.672399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.672765 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.674167 (+ 1402us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.674312 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.674578 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:52:21.675522 28070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: Neither blocks nor log segments found. Creating new log.
W20260923 20:52:21.679103 28089 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:52:21.679253 28089 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 20:52:21.679270 28070 log.cc:824] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:21.681998 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:21.685145 28070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: No bootstrap required, opened a new log
I20260923 20:52:21.685916 27896 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e3b00a0bf4794439bee90c7de413ffc9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41688
I20260923 20:52:21.686434 27896 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 20:52:21.686661 27896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 0 FOLLOWER]: Advancing to term 1
W20260923 20:52:21.689949 27875 tablet.cc:2369] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:52:21.677361 27854 master_runner.cc:186] Error getting master registration for 127.5.71.189:40001: OK, instance_id { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" instance_seqno: 1790196741034659 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 20:52:21.690254 27854 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 20:52:21.690713 27896 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.693384 27896 leader_election.cc:288] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [CANDIDATE]: Term 1 election: Requested vote from peers 9daf674a6a834cd194d0ae738f03e479 (127.5.71.189:40001), 490c519afbd44fb6ab4184a36626e5e8 (127.5.71.188:42055)
I20260923 20:52:21.698885 28070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.699348 28070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490c519afbd44fb6ab4184a36626e5e8, State: Initialized, Role: FOLLOWER
I20260923 20:52:21.699970 28070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.702674 28070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:21.702756 28045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3b00a0bf4794439bee90c7de413ffc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "490c519afbd44fb6ab4184a36626e5e8"
I20260923 20:52:21.703438 28045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:21.709651 28045 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3b00a0bf4794439bee90c7de413ffc9 in term 1.
I20260923 20:52:21.711268 27871 leader_election.cc:302] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490c519afbd44fb6ab4184a36626e5e8, e3b00a0bf4794439bee90c7de413ffc9; no voters:
I20260923 20:52:21.712083 28078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:21.702013 28094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.741379 28094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.744560 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.716766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.5.71.189:40001)
0923 20:52:21.717169 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.717178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.721157 (+ 3979us) server_negotiation.cc:407] Connection header received
0923 20:52:21.721731 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.721743 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.721860 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.721962 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.725381 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.726860 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.729546 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.730488 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.730776 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.735417 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.735427 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.735435 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.735508 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.740329 (+ 4821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.741383 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.741390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.741394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.741610 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.742169 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.742178 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.742185 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.742673 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.742955 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.743703 (+ 748us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.744039 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260923 20:52:21.758729 28078 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [term 1 LEADER]: Becoming Leader. State: Replica: e3b00a0bf4794439bee90c7de413ffc9, State: Running, Role: LEADER
I20260923 20:52:21.759658 28078 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.769977 27995 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } attempt: 5
I20260923 20:52:21.772375 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.716472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.5.71.190:46675)
0923 20:52:21.716682 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.716689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.745777 (+ 29088us) server_negotiation.cc:407] Connection header received
0923 20:52:21.746084 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.746092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.746203 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.746290 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.748685 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.750168 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.754658 (+ 4490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.755407 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.755540 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.762451 (+ 6911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.762459 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.762467 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.762552 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.766917 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.768521 (+ 1604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.768528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.768531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.768723 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.769603 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.769609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.769615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.770071 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.770386 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.771548 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.771860 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:52:21.772830 28099 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b00a0bf4794439bee90c7de413ffc9. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.773447 28099 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: This master's current role is: LEADER
Sep 23 20:52:21 dist-test-slave-dn29 krb5kdc[27841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196740, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:52:21.784659 28113 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 20:52:21.787302 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.716566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.5.71.188:42055)
0923 20:52:21.716983 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.716990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.752856 (+ 35866us) server_negotiation.cc:407] Connection header received
0923 20:52:21.753066 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.753073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.753174 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.753281 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.763239 (+ 9958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.764634 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.767012 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.767830 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.767952 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.775753 (+ 7801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.775763 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.775772 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.775880 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.780421 (+ 4541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.781718 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.781731 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.781738 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.782172 (+ 434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.783247 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.783254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.783286 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.784151 (+ 865us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.784798 (+ 647us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.786064 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.786829 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260923 20:52:21.790207 28112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:21.790764 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.770625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:42055 (local address 127.0.0.1:42882)
0923 20:52:21.771018 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:21.771037 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:21.771149 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:21.771901 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:21.771906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:21.772416 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:21.772625 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.772639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.774299 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.774303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.775656 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:21.775671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.776889 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.776895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:21.777111 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.777841 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:21.777894 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:21.781896 (+ 4002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:21.786753 (+ 4857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.786758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.786762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.787346 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.787851 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:21.787857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:21.787882 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:21.788217 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:21.789713 (+ 1496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:21.789733 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:21.789905 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 20:52:21.790044 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 20:52:21.790351 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
I20260923 20:52:21.792253 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:21.771382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.5.71.188:42055)
0923 20:52:21.771554 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.771560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:21.771578 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:52:21.771629 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:21.771633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:21.771706 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:21.772879 (+ 1173us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:21.773026 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.774078 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.775874 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:21.776553 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:21.776678 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:21.782143 (+ 5465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:21.782150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:21.782155 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:21.782195 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:21.786593 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.787494 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.787498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.787501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.787641 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:21.789145 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:21.789150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:21.789154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:21.789497 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:21.789629 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:21.791380 (+ 1751us) server_negotiation.cc:299] Negotiation successful
0923 20:52:21.791836 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 20:52:21.794070 28112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:21.798113 27995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: Bootstrap starting.
I20260923 20:52:21.802392 27995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:21.804059 27995 log.cc:824] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:21.814971 28045 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [term 1 FOLLOWER]: Refusing update from remote peer e3b00a0bf4794439bee90c7de413ffc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:52:21.816214 28099 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 20:52:21.818925 28024 tablet.cc:2369] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:52:21.819432 27995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: No bootstrap required, opened a new log
I20260923 20:52:21.792109 28003 master_runner.cc:186] Error getting master registration for 127.5.71.189:40001: OK, instance_id { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" instance_seqno: 1790196741034659 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 20:52:21.824193 28003 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 20:52:21.831318 27995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.831844 27995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9daf674a6a834cd194d0ae738f03e479, State: Initialized, Role: FOLLOWER
I20260923 20:52:21.832435 27995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } }
I20260923 20:52:21.835094 27995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:21.835645 27969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:21.839944 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3b00a0bf4794439bee90c7de413ffc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9daf674a6a834cd194d0ae738f03e479"
I20260923 20:52:21.842751 27969 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [term 1 FOLLOWER]: Refusing update from remote peer e3b00a0bf4794439bee90c7de413ffc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:52:21.845804 28094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b00a0bf4794439bee90c7de413ffc9. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.846145 28094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.850409 28099 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.850853 28099 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:21.851333 27970 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3b00a0bf4794439bee90c7de413ffc9 in term 1.
I20260923 20:52:21.853412 28112 catalog_manager.cc:1347] Generated new cluster ID: 3b536cf94f114e5f92c25e22649defef
I20260923 20:52:21.853528 28112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:21.860385 28094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.860697 28094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.834095 28117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.862177 28117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.862566 28078 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:52:21.899971 28112 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:21.901824 28112 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:21.904943 28117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b00a0bf4794439bee90c7de413ffc9. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.905381 28117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.905942 28133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.906487 28133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:21.908288 28099 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3b00a0bf4794439bee90c7de413ffc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b00a0bf4794439bee90c7de413ffc9" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 46675 } } peers { permanent_uuid: "9daf674a6a834cd194d0ae738f03e479" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 40001 } } peers { permanent_uuid: "490c519afbd44fb6ab4184a36626e5e8" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 42055 } } }
I20260923 20:52:21.908582 28099 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9 [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:21.922093 28112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3b00a0bf4794439bee90c7de413ffc9: Generated new TSK 0
I20260923 20:52:21.923286 28112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:21.924165 28135 catalog_manager.cc:1259] Loaded cluster ID: 3b536cf94f114e5f92c25e22649defef
I20260923 20:52:21.924331 28135 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: loading cluster ID for follower catalog manager: success
I20260923 20:52:21.927186 28135 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: acquiring CA information for follower catalog manager: success
I20260923 20:52:21.929898 28135 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9daf674a6a834cd194d0ae738f03e479: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 20:52:22.006479 28000 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:21.993712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.5.71.189:40001)
0923 20:52:21.993966 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.993975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.004435 (+ 10460us) server_negotiation.cc:407] Connection header received
0923 20:52:22.004764 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.004771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.004869 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.004949 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.006212 (+ 1263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58916: BlockingRecv error: recv got EOF from 127.0.0.1:58916 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260923 20:52:22.007378 28075 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:21.999955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.5.71.190:46675)
0923 20:52:22.000217 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.000227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.005429 (+ 5202us) server_negotiation.cc:407] Connection header received
0923 20:52:22.005849 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.005855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.005942 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.006017 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.007145 (+ 1128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41698: BlockingRecv error: recv got EOF from 127.0.0.1:41698 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260923 20:52:22.009265 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:21.993358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42894 (local address 127.5.71.188:42055)
0923 20:52:21.993591 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:21.993598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.004838 (+ 11240us) server_negotiation.cc:407] Connection header received
0923 20:52:22.007460 (+ 2622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.007469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.007560 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.007635 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.009011 (+ 1376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42894: BlockingRecv error: recv got EOF from 127.0.0.1:42894 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 20:52:22.022210 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.013890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.5.71.189:40001)
0923 20:52:22.014092 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.014099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.014597 (+ 498us) server_negotiation.cc:407] Connection header received
0923 20:52:22.014840 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.014846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.015071 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.015250 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.015941 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.017137 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.019133 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.019887 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.020020 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.020625 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.021477 (+ 852us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.021628 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.021756 (+ 128us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.021857 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260923 20:52:22.032461 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.030420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.5.71.188:42055)
0923 20:52:22.030602 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.030607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.030675 (+ 68us) server_negotiation.cc:407] Connection header received
0923 20:52:22.031037 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.031042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.031154 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.031299 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.032220 (+ 921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42898: BlockingRecv error: recv got EOF from 127.0.0.1:42898 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 20:52:22.051393 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.037669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.5.71.189:40001)
0923 20:52:22.037914 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.037923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.038514 (+ 591us) server_negotiation.cc:407] Connection header received
0923 20:52:22.038749 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.038759 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.038908 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.039019 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.039470 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.040651 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.048894 (+ 8243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.049712 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.049824 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.050492 (+ 668us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.050664 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.050731 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.050966 (+ 235us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.051056 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260923 20:52:22.085995 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.077772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.5.71.188:42055)
0923 20:52:22.078022 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.078032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.084502 (+ 6470us) server_negotiation.cc:407] Connection header received
0923 20:52:22.084748 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.084754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.084842 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.084958 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.085751 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42914: BlockingRecv error: recv got EOF from 127.0.0.1:42914 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260923 20:52:22.096927 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.091209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.5.71.189:40001)
0923 20:52:22.091484 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.091492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.092049 (+ 557us) server_negotiation.cc:407] Connection header received
0923 20:52:22.092261 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.092268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.092350 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.092456 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.092875 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.093636 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.094995 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.095468 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.095619 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.096089 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.096212 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.096320 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.096460 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.096558 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260923 20:52:22.109953 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.103198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.5.71.188:42055)
0923 20:52:22.103439 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.103448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.108486 (+ 5038us) server_negotiation.cc:407] Connection header received
0923 20:52:22.108722 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.108728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.108815 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.108928 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.109713 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42930: BlockingRecv error: recv got EOF from 127.0.0.1:42930 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:52:22.123921 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.117601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.5.71.189:40001)
0923 20:52:22.117877 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.117887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.118473 (+ 586us) server_negotiation.cc:407] Connection header received
0923 20:52:22.118729 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.118735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.118823 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.118947 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.119396 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.120382 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.121928 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.122453 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.122610 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.123055 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.123175 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.123289 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.123452 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.123542 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260923 20:52:22.144075 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.141493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42934 (local address 127.5.71.188:42055)
0923 20:52:22.141885 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.141895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.142472 (+ 577us) server_negotiation.cc:407] Connection header received
0923 20:52:22.142713 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.142719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.142812 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.142939 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.143859 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42934: BlockingRecv error: recv got EOF from 127.0.0.1:42934 (error 108)
Metrics: {"server-negotiator.queue_time_us":210}
I20260923 20:52:22.162509 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.156614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.5.71.189:40001)
0923 20:52:22.156908 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.156917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.157486 (+ 569us) server_negotiation.cc:407] Connection header received
0923 20:52:22.157707 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.157712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.157793 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.157900 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.158367 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.159111 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.160519 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.160988 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.161131 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.161608 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.161777 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.161833 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.162033 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.162116 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260923 20:52:22.180730 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.178388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.5.71.188:42055)
0923 20:52:22.178618 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.178628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.179240 (+ 612us) server_negotiation.cc:407] Connection header received
0923 20:52:22.179453 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.179459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.179547 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.179664 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.180466 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42940: BlockingRecv error: recv got EOF from 127.0.0.1:42940 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:52:22.206317 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.200130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.5.71.189:40001)
0923 20:52:22.200371 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.200381 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.200936 (+ 555us) server_negotiation.cc:407] Connection header received
0923 20:52:22.201161 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.201173 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.201252 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.201358 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.201803 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.202782 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.204440 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.204868 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.205012 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.205482 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.205620 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.205731 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.205895 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.205981 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260923 20:52:22.214545 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.212784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.5.71.188:42055)
0923 20:52:22.212999 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.213007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.213065 (+ 58us) server_negotiation.cc:407] Connection header received
0923 20:52:22.213352 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.213358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.213431 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.213545 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.214293 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42946: BlockingRecv error: recv got EOF from 127.0.0.1:42946 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 20:52:22.256340 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.250404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.5.71.189:40001)
0923 20:52:22.250672 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.250681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.251241 (+ 560us) server_negotiation.cc:407] Connection header received
0923 20:52:22.251505 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.251509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.251618 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.251743 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.252203 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.253074 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.254395 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.254880 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.255028 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.255494 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.255603 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.255705 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.255902 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.255983 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260923 20:52:22.265110 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.262889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.5.71.188:42055)
0923 20:52:22.263117 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.263126 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.263673 (+ 547us) server_negotiation.cc:407] Connection header received
0923 20:52:22.263907 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.263913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.263994 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.264105 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.264875 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42952: BlockingRecv error: recv got EOF from 127.0.0.1:42952 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 20:52:22.340122 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.333208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.5.71.189:40001)
0923 20:52:22.333477 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.333486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.333930 (+ 444us) server_negotiation.cc:407] Connection header received
0923 20:52:22.334146 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.334151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.334218 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.334321 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.334892 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.335760 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.337943 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.338594 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.338693 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.339246 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.339381 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.339452 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.339646 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.339715 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
W20260923 20:52:22.349885 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.347247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.5.71.188:42055)
0923 20:52:22.347561 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.347568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.348007 (+ 439us) server_negotiation.cc:407] Connection header received
0923 20:52:22.348244 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.348250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.348332 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.348429 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.349646 (+ 1217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42954: BlockingRecv error: recv got EOF from 127.0.0.1:42954 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 20:52:22.486913 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.481300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.5.71.189:40001)
0923 20:52:22.481566 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.481573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.481825 (+ 252us) server_negotiation.cc:407] Connection header received
0923 20:52:22.481990 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.481994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.482065 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.482146 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.482610 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.483396 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.484949 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.485451 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.485561 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.486087 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.486208 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.486281 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.486468 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.486541 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260923 20:52:22.495543 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.493428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42966 (local address 127.5.71.188:42055)
0923 20:52:22.493681 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.493688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.494004 (+ 316us) server_negotiation.cc:407] Connection header received
0923 20:52:22.494190 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.494194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.494319 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.494409 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.495275 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42966: BlockingRecv error: recv got EOF from 127.0.0.1:42966 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260923 20:52:22.762034 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:22.756510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.5.71.189:40001)
0923 20:52:22.756858 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.756863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.756967 (+ 104us) server_negotiation.cc:407] Connection header received
0923 20:52:22.757196 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.757202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.757276 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.757369 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:22.757893 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.758753 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.760215 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:22.760670 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:22.760804 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:22.761211 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:22.761327 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:22.761396 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:22.761636 (+ 240us) server_negotiation.cc:299] Negotiation successful
0923 20:52:22.761751 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260923 20:52:22.772687 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:22.770157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.5.71.188:42055)
0923 20:52:22.770402 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:22.770409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:22.770780 (+ 371us) server_negotiation.cc:407] Connection header received
0923 20:52:22.771014 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:22.771021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:22.771140 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:22.771289 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:22.772423 (+ 1134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42970: BlockingRecv error: recv got EOF from 127.0.0.1:42970 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260923 20:52:22.789178 28113 catalog_manager.cc:1259] Loaded cluster ID: 3b536cf94f114e5f92c25e22649defef
I20260923 20:52:22.789295 28113 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: loading cluster ID for follower catalog manager: success
I20260923 20:52:22.792037 28113 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: acquiring CA information for follower catalog manager: success
I20260923 20:52:22.794215 28113 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 490c519afbd44fb6ab4184a36626e5e8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 20:52:23.295409 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:23.289672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58996 (local address 127.5.71.189:40001)
0923 20:52:23.290233 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:23.290238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:23.290354 (+ 116us) server_negotiation.cc:407] Connection header received
0923 20:52:23.290549 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:23.290553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:23.290635 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:23.290726 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:23.291151 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.291908 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.293614 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.294000 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.294128 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:23.294569 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:23.294675 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:23.294762 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:23.294943 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 20:52:23.295010 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":277,"threads_started":1}
I20260923 20:52:23.308606 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:23.302330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.5.71.188:42055)
0923 20:52:23.302908 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:23.302913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:23.303044 (+ 131us) server_negotiation.cc:407] Connection header received
0923 20:52:23.303206 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:23.303210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:23.303401 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:23.303514 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:23.304201 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.304993 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.306311 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.306704 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.306811 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:23.307350 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:23.308070 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:23.308180 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:23.308315 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 20:52:23.308383 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":280,"threads_started":1}
I20260923 20:52:23.328980 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:23.315855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.5.71.189:40001)
0923 20:52:23.316046 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:23.316053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:23.322544 (+ 6491us) server_negotiation.cc:407] Connection header received
0923 20:52:23.323402 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:23.323409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:23.323511 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:23.323583 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:23.324484 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.325353 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.326947 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.327356 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.327502 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:23.328088 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:23.328202 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:23.328313 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:23.328434 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 20:52:23.328518 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 20:52:23.331465 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:23.316001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.5.71.188:42055)
0923 20:52:23.321566 (+ 5565us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:23.321578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:23.324800 (+ 3222us) server_negotiation.cc:407] Connection header received
0923 20:52:23.324902 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:23.324909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:23.325009 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:23.325123 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:23.325752 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.326827 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.329276 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.329849 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.329978 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:23.330502 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:23.330646 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:23.330718 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:23.331017 (+ 299us) server_negotiation.cc:299] Negotiation successful
0923 20:52:23.331113 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5339}
I20260923 20:52:23.331979 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:23.315546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.5.71.190:46675)
0923 20:52:23.323667 (+ 8121us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:23.323674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:23.323693 (+ 19us) server_negotiation.cc:407] Connection header received
0923 20:52:23.323854 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:23.323859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:23.324103 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:23.324319 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:23.325285 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.326560 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.329058 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:23.329650 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:23.329804 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:23.330391 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:23.330867 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:23.330969 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:23.331546 (+ 577us) server_negotiation.cc:299] Negotiation successful
0923 20:52:23.331650 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8003,"thread_start_us":7890,"threads_started":1}
I20260923 20:52:23.338160 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27854
I20260923 20:52:23.339751 27916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:23.454502 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27854
I20260923 20:52:23.476402 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 27928
W20260923 20:52:23.476359 27946 connection.cc:535] server connection from 127.0.0.1:58890 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 20:52:23.477581 27990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:23.595616 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 27928
I20260923 20:52:23.616473 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 28003
I20260923 20:52:23.617748 28065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:23.726828 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 28003
2026-09-23T20:52:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5182 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 20:52:23 dist-test-slave-dn29 krb5kdc[28273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 20:52:23 dist-test-slave-dn29 krb5kdc[28273](info): set up 2 sockets
Sep 23 20:52:23 dist-test-slave-dn29 krb5kdc[28273](info): commencing operation
krb5kdc: starting...
W20260923 20:52:24.888840 5406 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/test-user.keytab.
Sep 23 20:52:25 dist-test-slave-dn29 krb5kdc[28273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T20:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T20:52:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.71.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
Entry for principal kudu/127.5.71.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab.
I20260923 20:52:25.180013 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:41735
--webserver_interface=127.5.71.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:41735,127.5.71.189:36897,127.5.71.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 20:52:25.526051 28286 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 20:52:25.526494 28286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:25.526644 28286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:25.526789 28286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:25.534307 28286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:25.534468 28286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:25.534595 28286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:25.534695 28286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:25.534789 28286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:25.545460 28286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:41735,127.5.71.189:36897,127.5.71.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.190.keytab
--principal=kudu/127.5.71.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.190:41735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.71.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:25.546923 28286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:25.548432 28286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:25.561364 28294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:25.561339 28292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:25.561626 28291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:25.563164 28286 server_base.cc:1034] running on GCE node
I20260923 20:52:25.564394 28286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:25.566052 28286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:25.567345 28286 hybrid_clock.cc:648] HybridClock initialized: now 1790196745567296 us; error 55 us; skew 500 ppm
I20260923 20:52:25.567704 28286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:25 dist-test-slave-dn29 krb5kdc[28273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:25.571141 28286 init.cc:377] Logged in from keytab as kudu/127.5.71.190@KRBTEST.COM (short username kudu)
I20260923 20:52:25.572849 28286 webserver.cc:460] Webserver started at http://127.5.71.190:40403/ using document root <none> and password file <none>
I20260923 20:52:25.573622 28286 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:25.573762 28286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:25.574095 28286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:25.577379 28286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data/instance:
uuid: "587e49f036e94359b14c75527d0c720e"
format_stamp: "Formatted at 2026-09-23 20:52:25 on dist-test-slave-dn29"
server_key: "41ee1f33ebf12c5f5528ad643f1350c5"
server_key_iv: "abc99d9021e4feea91aae4dbdf7a4cff"
server_key_version: "encryptionkey@0"
I20260923 20:52:25.578174 28286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal/instance:
uuid: "587e49f036e94359b14c75527d0c720e"
format_stamp: "Formatted at 2026-09-23 20:52:25 on dist-test-slave-dn29"
server_key: "41ee1f33ebf12c5f5528ad643f1350c5"
server_key_iv: "abc99d9021e4feea91aae4dbdf7a4cff"
server_key_version: "encryptionkey@0"
I20260923 20:52:25.583719 28286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260923 20:52:25.587461 28301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:25.589285 28286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:25.589516 28286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal
uuid: "587e49f036e94359b14c75527d0c720e"
format_stamp: "Formatted at 2026-09-23 20:52:25 on dist-test-slave-dn29"
server_key: "41ee1f33ebf12c5f5528ad643f1350c5"
server_key_iv: "abc99d9021e4feea91aae4dbdf7a4cff"
server_key_version: "encryptionkey@0"
I20260923 20:52:25.589787 28286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:25.604488 28286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:25.609418 28286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:25.609750 28286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:25.630298 28286 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.190:41735
I20260923 20:52:25.630337 28352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.190:41735 every 8 connection(s)
I20260923 20:52:25.631865 28286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/data/info.pb
I20260923 20:52:25.636204 28353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:25.638317 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 28286
I20260923 20:52:25.638532 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-0/wal/instance
I20260923 20:52:25.639127 5406 external_mini_cluster.cc:1428] Setting key 6bc43519c1db06757f02874e15397aef
I20260923 20:52:25.647261 28353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:25 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.189@KRBTEST.COM" created.
I20260923 20:52:25.674458 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.652064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41735 (local address 127.0.0.1:37952)
0923 20:52:25.653381 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.653418 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:25.655419 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:25.656411 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:25.656418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:25.657246 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:25.657613 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:25.657635 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:25.659384 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:25.659389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:25.660681 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:25.660694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:25.661739 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:25.661746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:25.661973 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:25.662756 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:25.662797 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:25.665278 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:25.669246 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:25.669260 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:25.669274 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:25.669957 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:25.670525 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:25.670529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:25.670532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:25.670768 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:25.672056 (+ 1288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:25.672062 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:25.672307 (+ 245us) client_negotiation.cc:769] Sending connection context
0923 20:52:25.672532 (+ 225us) client_negotiation.cc:240] Negotiation successful
0923 20:52:25.672968 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"spinlock_wait_cycles":37760,"thread_start_us":838,"threads_started":1}
I20260923 20:52:25.674486 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.652094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.5.71.190:41735)
0923 20:52:25.653069 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:25.653091 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:25.653557 (+ 466us) server_negotiation.cc:407] Connection header received
0923 20:52:25.655881 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:25.655888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:25.656154 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:25.656388 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:25.657864 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:25.659208 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:25.660835 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:25.661464 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:25.661714 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:25.665540 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:25.665550 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:25.665565 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:25.665596 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:25.669075 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:25.670164 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:25.670175 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:25.670179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:25.670388 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:25.670913 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:25.670921 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:25.670927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:25.671879 (+ 952us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:25.672102 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:25.672749 (+ 647us) server_negotiation.cc:299] Negotiation successful
0923 20:52:25.673177 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":701,"thread_start_us":466,"threads_started":1}
I20260923 20:52:25.678095 28353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:25.679421 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.678924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43654)
0923 20:52:25.679138 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.679269 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260923 20:52:25.679847 28303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
I20260923 20:52:25.681627 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.681188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43666)
0923 20:52:25.681357 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.681448 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 20:52:25.681916 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:36897: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab.
Entry for principal kudu/127.5.71.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab.
I20260923 20:52:25.708706 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:36897
--webserver_interface=127.5.71.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab
--principal=kudu/127.5.71.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:41735,127.5.71.189:36897,127.5.71.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:52:25.731321 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } attempt: 1
I20260923 20:52:25.732709 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.732176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43680)
0923 20:52:25.732428 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.732530 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260923 20:52:25.734006 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.733567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43682)
0923 20:52:25.733743 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.733846 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260923 20:52:25.734205 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:36897: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
I20260923 20:52:25.802505 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } attempt: 2
I20260923 20:52:25.804077 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.803365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43684)
0923 20:52:25.803691 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.803868 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260923 20:52:25.805335 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.804859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43690)
0923 20:52:25.805030 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.805157 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260923 20:52:25.805563 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:36897: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
I20260923 20:52:25.896842 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } attempt: 3
I20260923 20:52:25.898236 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.897577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43698)
0923 20:52:25.897869 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.898009 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260923 20:52:25.899585 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:25.899069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43706)
0923 20:52:25.899266 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:25.899378 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260923 20:52:25.899865 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:36897: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
I20260923 20:52:26.043160 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } attempt: 4
I20260923 20:52:26.044682 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.043956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43714)
0923 20:52:26.044382 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.044503 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260923 20:52:26.045905 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.045445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43724)
0923 20:52:26.045634 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.045744 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260923 20:52:26.046216 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.189:36897: Network error: Client connection negotiation failed: client connection to 127.5.71.189:36897: connect: Connection refused (error 111)
W20260923 20:52:26.070281 28360 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 20:52:26.070664 28360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:26.070739 28360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:26.070837 28360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:26.079105 28360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:26.079217 28360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:26.079275 28360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:26.079321 28360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:26.079385 28360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:26.090138 28360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:41735,127.5.71.189:36897,127.5.71.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.189.keytab
--principal=kudu/127.5.71.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.189:36897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.71.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:26.091521 28360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:26.092960 28360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:26.106458 28368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:26.106575 28366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:26.106899 28365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:26.107645 28360 server_base.cc:1034] running on GCE node
I20260923 20:52:26.108618 28360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:26.110175 28360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:26.111449 28360 hybrid_clock.cc:648] HybridClock initialized: now 1790196746111373 us; error 34 us; skew 500 ppm
I20260923 20:52:26.111893 28360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:26.115394 28360 init.cc:377] Logged in from keytab as kudu/127.5.71.189@KRBTEST.COM (short username kudu)
I20260923 20:52:26.117208 28360 webserver.cc:460] Webserver started at http://127.5.71.189:43915/ using document root <none> and password file <none>
I20260923 20:52:26.117929 28360 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:26.118017 28360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:26.118285 28360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:26.121388 28360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data/instance:
uuid: "0a7190f6535546cc9ebd48a3a4d6589b"
format_stamp: "Formatted at 2026-09-23 20:52:26 on dist-test-slave-dn29"
server_key: "9a3b380bdde074b7e29582d80eb9d6fe"
server_key_iv: "3e0d495156666bf94e4519b18c0d95ba"
server_key_version: "encryptionkey@0"
I20260923 20:52:26.122198 28360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal/instance:
uuid: "0a7190f6535546cc9ebd48a3a4d6589b"
format_stamp: "Formatted at 2026-09-23 20:52:26 on dist-test-slave-dn29"
server_key: "9a3b380bdde074b7e29582d80eb9d6fe"
server_key_iv: "3e0d495156666bf94e4519b18c0d95ba"
server_key_version: "encryptionkey@0"
I20260923 20:52:26.127584 28360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260923 20:52:26.131289 28375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:26.132867 28360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:26.133067 28360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal
uuid: "0a7190f6535546cc9ebd48a3a4d6589b"
format_stamp: "Formatted at 2026-09-23 20:52:26 on dist-test-slave-dn29"
server_key: "9a3b380bdde074b7e29582d80eb9d6fe"
server_key_iv: "3e0d495156666bf94e4519b18c0d95ba"
server_key_version: "encryptionkey@0"
I20260923 20:52:26.133318 28360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:26.172849 28360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:26.177825 28360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:26.178117 28360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:26.197424 28360 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.189:36897
I20260923 20:52:26.197520 28426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.189:36897 every 8 connection(s)
I20260923 20:52:26.198925 28360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/data/info.pb
I20260923 20:52:26.203132 28427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:26.208012 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 28360
I20260923 20:52:26.208245 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-1/wal/instance
I20260923 20:52:26.208701 5406 external_mini_cluster.cc:1428] Setting key b0111221f7ca5e9dc8bfa8f22493fcd4
I20260923 20:52:26.210580 28427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.71.188@KRBTEST.COM; defaulting to no policy
I20260923 20:52:26.234251 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.213595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37958 (local address 127.5.71.190:41735)
0923 20:52:26.214324 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.214329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.215408 (+ 1079us) server_negotiation.cc:407] Connection header received
0923 20:52:26.217651 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.217658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.217762 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.217894 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.219559 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.220483 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.222185 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.222655 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.222814 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.227105 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.227111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.227114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.227173 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.230597 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.231520 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.231540 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.231543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.231691 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.232230 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.232239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.232243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.232564 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.232761 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.233565 (+ 804us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.233830 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":368,"threads_started":1}
I20260923 20:52:26.235430 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.214274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41735 (local address 127.0.0.1:37958)
0923 20:52:26.215216 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.215266 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.217307 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.218162 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.218172 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.218750 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.219323 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.219342 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.220670 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.220674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.221991 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.222004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.222918 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.222926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.223185 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.224073 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.224099 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.226838 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.230802 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.230813 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.230833 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.231361 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.231858 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.231864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.231868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.232087 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.232717 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.232726 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.233142 (+ 416us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.233445 (+ 303us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.234047 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":447,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.71.188@KRBTEST.COM" created.
I20260923 20:52:26.237599 28427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:26.254230 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.238573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43728)
0923 20:52:26.238805 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.238822 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.238919 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.240012 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.240017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.240519 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.240662 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.240672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.242220 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.242229 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.243240 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.243252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.244266 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.244272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.244482 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.245145 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.245164 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.247642 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.251390 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.251398 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.251403 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.251868 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.252295 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.252301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.252305 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.252454 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.253124 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.253132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.253309 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.253462 (+ 153us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.253765 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260923 20:52:26.254295 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.238770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.5.71.189:36897)
0923 20:52:26.239503 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.239509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.239537 (+ 28us) server_negotiation.cc:407] Connection header received
0923 20:52:26.239583 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.239589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.239850 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.240026 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.240852 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.242066 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.243411 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.244104 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.244254 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.247976 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.247989 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.248006 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.248037 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.251223 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.252017 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.252034 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.252037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.252167 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.252568 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.252576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.252579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.252969 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.253205 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.253642 (+ 437us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.253876 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":420,"threads_started":1}
I20260923 20:52:26.257107 28427 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:26.258589 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.258025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57630)
0923 20:52:26.258276 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.258393 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260923 20:52:26.258926 28376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.260998 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.260429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57632)
0923 20:52:26.260657 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.260808 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260923 20:52:26.261307 28427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.71.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab.
Entry for principal kudu/127.5.71.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab.
I20260923 20:52:26.276593 5406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQIJ804/build/asan/bin/kudu
/tmp/dist-test-taskQIJ804/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:40785
--webserver_interface=127.5.71.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab
--principal=kudu/127.5.71.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.71.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.71.190:41735,127.5.71.189:36897,127.5.71.188:40785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 20:52:26.310701 28427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 1
I20260923 20:52:26.312021 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.311449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57648)
0923 20:52:26.311688 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.311832 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260923 20:52:26.313169 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.312774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57650)
0923 20:52:26.312943 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.313044 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260923 20:52:26.313426 28427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.345494 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } attempt: 5
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:26.361660 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.346445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.5.71.189:36897)
0923 20:52:26.346643 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.346648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.346920 (+ 272us) server_negotiation.cc:407] Connection header received
0923 20:52:26.347114 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.347118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.347211 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.347320 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.348419 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.349328 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.351125 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.351628 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.351740 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.355563 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.355569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.355573 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.355616 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.358588 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.359353 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.359360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.359364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.359500 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.359977 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.359985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.359989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.360317 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.360540 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.361044 (+ 504us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.361295 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 20:52:26.361666 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.346341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43734)
0923 20:52:26.346755 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.346783 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.346961 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.347398 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.347403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.347984 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.348231 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.348249 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.349478 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.349483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.350959 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.350976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.351827 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.351834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.352035 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.352728 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.352752 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.355314 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.358765 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.358770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.358775 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.359206 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.359634 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.359639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.359643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.359848 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.360497 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.360503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.360735 (+ 232us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.360919 (+ 184us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.361261 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260923 20:52:26.362903 28353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:26.364231 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.363643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57658)
0923 20:52:26.363963 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.364054 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260923 20:52:26.365271 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.364904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57660)
0923 20:52:26.365059 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.365140 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260923 20:52:26.365481 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.381682 28427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 2
I20260923 20:52:26.383019 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.382441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57666)
0923 20:52:26.382731 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.382840 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260923 20:52:26.384315 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.383932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57668)
0923 20:52:26.384096 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.384188 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260923 20:52:26.384565 28427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.416785 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 1
I20260923 20:52:26.418187 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.417608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57680)
0923 20:52:26.417904 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.418009 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260923 20:52:26.419267 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.418912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57692)
0923 20:52:26.419068 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.419155 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260923 20:52:26.419473 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.475878 28427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 3
I20260923 20:52:26.477214 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.476675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57694)
0923 20:52:26.476929 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.477030 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260923 20:52:26.478462 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.477939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57710)
0923 20:52:26.478225 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.478326 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260923 20:52:26.478809 28427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.487984 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 2
I20260923 20:52:26.489456 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.488832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57724)
0923 20:52:26.489124 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.489274 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260923 20:52:26.490808 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.490310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57736)
0923 20:52:26.490560 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.490677 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260923 20:52:26.491052 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.597306 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 3
I20260923 20:52:26.598567 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.598041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57746)
0923 20:52:26.598294 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.598387 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260923 20:52:26.599936 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.599320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57748)
0923 20:52:26.599677 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.599758 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
W20260923 20:52:26.600131 28353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
I20260923 20:52:26.622077 28427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 4
I20260923 20:52:26.623253 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.622764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57764)
0923 20:52:26.623012 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.623098 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260923 20:52:26.624339 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.623984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57766)
0923 20:52:26.624141 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.624232 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 20:52:26.624538 28427 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.71.188:40785: Network error: Client connection negotiation failed: client connection to 127.5.71.188:40785: connect: Connection refused (error 111)
W20260923 20:52:26.636209 28435 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 20:52:26.636595 28435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 20:52:26.636749 28435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 20:52:26.636888 28435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 20:52:26.645022 28435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 20:52:26.645123 28435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 20:52:26.645215 28435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 20:52:26.645291 28435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 20:52:26.645406 28435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 20:52:26.655862 28435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.71.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.71.190:41735,127.5.71.189:36897,127.5.71.188:40785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/krb5kdc/kudu_127.5.71.188.keytab
--principal=kudu/127.5.71.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.71.188:40785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.71.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 23 Sep 2026 19:58:29 UTC on ccycloud.quasar-janqnj.root.comops.site
build id 4668599
ASAN enabled
I20260923 20:52:26.657305 28435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 20:52:26.658740 28435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 20:52:26.671327 28443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:26.671371 28441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 20:52:26.671394 28440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 20:52:26.672194 28435 server_base.cc:1034] running on GCE node
I20260923 20:52:26.673259 28435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 20:52:26.674988 28435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 20:52:26.676223 28435 hybrid_clock.cc:648] HybridClock initialized: now 1790196746676131 us; error 56 us; skew 500 ppm
I20260923 20:52:26.676615 28435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 20:52:26.685985 28435 init.cc:377] Logged in from keytab as kudu/127.5.71.188@KRBTEST.COM (short username kudu)
I20260923 20:52:26.687755 28435 webserver.cc:460] Webserver started at http://127.5.71.188:42981/ using document root <none> and password file <none>
I20260923 20:52:26.688566 28435 fs_manager.cc:359] Metadata directory not provided
I20260923 20:52:26.688714 28435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 20:52:26.689057 28435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 20:52:26.692195 28435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data/instance:
uuid: "a1460ff13cfe410ea0458ae9774d957f"
format_stamp: "Formatted at 2026-09-23 20:52:26 on dist-test-slave-dn29"
server_key: "665936e8554388fc72a3368e87995279"
server_key_iv: "e1aafea8eed2cb59aa4e39facaf3af67"
server_key_version: "encryptionkey@0"
I20260923 20:52:26.693001 28435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal/instance:
uuid: "a1460ff13cfe410ea0458ae9774d957f"
format_stamp: "Formatted at 2026-09-23 20:52:26 on dist-test-slave-dn29"
server_key: "665936e8554388fc72a3368e87995279"
server_key_iv: "e1aafea8eed2cb59aa4e39facaf3af67"
server_key_version: "encryptionkey@0"
I20260923 20:52:26.698696 28435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260923 20:52:26.702666 28450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 20:52:26.704247 28435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260923 20:52:26.704454 28435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data,/tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal
uuid: "a1460ff13cfe410ea0458ae9774d957f"
format_stamp: "Formatted at 2026-09-23 20:52:26 on dist-test-slave-dn29"
server_key: "665936e8554388fc72a3368e87995279"
server_key_iv: "e1aafea8eed2cb59aa4e39facaf3af67"
server_key_version: "encryptionkey@0"
I20260923 20:52:26.704700 28435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 20:52:26.717367 28435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 20:52:26.722203 28435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 20:52:26.722532 28435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 20:52:26.743105 28435 rpc_server.cc:307] RPC server started. Bound to: 127.5.71.188:40785
I20260923 20:52:26.743129 28501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.71.188:40785 every 8 connection(s)
I20260923 20:52:26.744766 28435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/data/info.pb
I20260923 20:52:26.748848 28502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 20:52:26.754889 5406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQIJ804/build/asan/bin/kudu as pid 28435
I20260923 20:52:26.755100 5406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQIJ804/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790196359367181-5406-0/minicluster-data/master-2/wal/instance
I20260923 20:52:26.755640 5406 external_mini_cluster.cc:1428] Setting key 4c731cc27f69a2d658891ca4adb37853
I20260923 20:52:26.755816 28502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:52:26.777320 28353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 4
I20260923 20:52:26.777494 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.758752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.5.71.190:41735)
0923 20:52:26.759306 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.759312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.759984 (+ 672us) server_negotiation.cc:407] Connection header received
0923 20:52:26.761179 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.761184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.761273 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.761444 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.762847 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.763741 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.765646 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.766388 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.766546 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.770750 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.770756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.770759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.770795 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.773930 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.774894 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.774900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.774903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.775040 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.775486 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.775491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.775494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.775776 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.775991 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.776812 (+ 821us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.777040 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":229,"threads_started":1}
I20260923 20:52:26.778944 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.758772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.190:41735 (local address 127.0.0.1:37960)
0923 20:52:26.759826 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.759871 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.760852 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.761530 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.761537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.762257 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.762645 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.762663 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.763961 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.763969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.765468 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.765483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.766628 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.766635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.766904 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.767784 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.767884 (+ 100us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.770519 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.774074 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.774089 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.774103 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.774748 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.775173 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.775178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.775181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.775368 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.775961 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.775970 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.776406 (+ 436us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.776731 (+ 325us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.777931 (+ 1200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":22,"thread_start_us":496,"threads_started":1}
I20260923 20:52:26.778978 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.757622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.5.71.188:40785)
0923 20:52:26.758368 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.758378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.758413 (+ 35us) server_negotiation.cc:407] Connection header received
0923 20:52:26.760852 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.760881 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.761213 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.761366 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.763170 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.764793 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.766512 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.767308 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.767447 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.771636 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.771645 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.771660 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.771695 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.774563 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.775308 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.775316 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.775318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.775444 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.775864 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.775871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.775874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.776398 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.776729 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.777053 (+ 324us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.777408 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":556,"thread_start_us":362,"threads_started":1}
I20260923 20:52:26.780938 28502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.190@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.190@KRBTEST.COM
I20260923 20:52:26.796136 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.777933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57786)
0923 20:52:26.778214 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.778235 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.778363 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.779681 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.779686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.780188 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.780367 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.780376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.782466 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.782472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.784127 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.784145 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.785114 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.785120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.785324 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.785977 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.786002 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.788976 (+ 2974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.793041 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.793051 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.793055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.793570 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.793945 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.793950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.793955 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.794153 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.795107 (+ 954us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.795111 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.795238 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.795385 (+ 147us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.795701 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260923 20:52:26.796168 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.777996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.5.71.188:40785)
0923 20:52:26.778953 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.778956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.778977 (+ 21us) server_negotiation.cc:407] Connection header received
0923 20:52:26.779015 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.779018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.779127 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.779657 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.780543 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.782307 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.784265 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.784953 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.785082 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.789232 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.789241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.789246 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.789283 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.792901 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.793712 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.793717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.793719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.793829 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.794294 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.794300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.794305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.794677 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.794842 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.795493 (+ 651us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.795707 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":652,"threads_started":1}
I20260923 20:52:26.796872 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.781823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.189:36897 (local address 127.0.0.1:43746)
0923 20:52:26.781957 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.781975 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.782068 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.782399 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.782403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.782906 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.783111 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.783125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.784247 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.784252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.785480 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.785490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.786378 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.786384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.786613 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.787211 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.787236 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.789968 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.793976 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.793981 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.793985 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.794479 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.794892 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.794897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.794901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.795092 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.795856 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.795861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.796039 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.796217 (+ 178us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.796486 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 20:52:26.797772 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.781769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.5.71.189:36897)
0923 20:52:26.781951 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.781957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.782035 (+ 78us) server_negotiation.cc:407] Connection header received
0923 20:52:26.782194 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.782200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.782276 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.782378 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.783292 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.784065 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.785611 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.786214 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.786342 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.790211 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.790218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.790225 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.790265 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.793784 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.794600 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.794608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.794612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.794788 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.795237 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.795243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.795246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.795610 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.795769 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.797153 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.797392 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 20:52:26.798748 28502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } has no permanent_uuid. Determining permanent_uuid...
I20260923 20:52:26.800443 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.782728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.5.71.190:41735)
0923 20:52:26.782899 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.782904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.782921 (+ 17us) server_negotiation.cc:407] Connection header received
0923 20:52:26.782969 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.782973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.783081 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.783188 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.784689 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.785540 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.787453 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.787949 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.788073 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.792777 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.792782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.792785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.792814 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.796704 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.797538 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.797543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.797546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.797712 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.798119 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.798124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.798127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.798485 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.798780 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.799869 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.800105 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 20:52:26.803092 28353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e: Bootstrap starting.
I20260923 20:52:26.806834 28353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e: Neither blocks nor log segments found. Creating new log.
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.188@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:52:26.808243 28353 log.cc:824] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e: Log is configured to *not* fsync() on all Append() calls
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.71.189@KRBTEST.COM
I20260923 20:52:26.812299 28353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e: No bootstrap required, opened a new log
I20260923 20:52:26.815456 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.799434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57790)
0923 20:52:26.799879 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.799902 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.800010 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.800321 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.800325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.800788 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.801028 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.801041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.802262 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.802270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.803480 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.803496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.804421 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.804429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.804693 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.805374 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.805397 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.808150 (+ 2753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.812323 (+ 4173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.812328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.812332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.812824 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.813271 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.813276 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.813279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.813433 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.814436 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.814443 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.814598 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.814762 (+ 164us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.815066 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260923 20:52:26.816326 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.799438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57790 (local address 127.5.71.188:40785)
0923 20:52:26.799592 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.799610 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.799960 (+ 350us) server_negotiation.cc:407] Connection header received
0923 20:52:26.800129 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.800134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.800210 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.800307 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.801204 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.802142 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.803638 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.804262 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.804390 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.808403 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.808410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.808414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.808451 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.812136 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.812956 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.812962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.812965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.813141 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.813833 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.813839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.813843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.814296 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.814449 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.815755 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.815998 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 20:52:26.819342 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.802613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.5.71.189:36897)
0923 20:52:26.802774 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.802779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.802797 (+ 18us) server_negotiation.cc:407] Connection header received
0923 20:52:26.802848 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.802852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.802921 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.803022 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.804510 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.805619 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.807371 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.808018 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.808159 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.812868 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.812873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.812876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.812939 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.816209 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.817052 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.817059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.817062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.817256 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.817765 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.817769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.817772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.818136 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.818413 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.818751 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.819002 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 20:52:26.822068 28502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: Bootstrap starting.
I20260923 20:52:26.822252 28353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:26.822681 28353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 587e49f036e94359b14c75527d0c720e, State: Initialized, Role: FOLLOWER
I20260923 20:52:26.823218 28353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:26.824826 5406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 20:52:26.825413 28353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:26.826109 28502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:26.827487 28502 log.cc:824] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:26.828158 28328 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "587e49f036e94359b14c75527d0c720e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37966
I20260923 20:52:26.828029 28510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:26.828590 28328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 20:52:26.828644 28510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:26.828790 28328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:26.830986 28328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:26.831133 28502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: No bootstrap required, opened a new log
I20260923 20:52:26.832350 28328 leader_election.cc:288] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [CANDIDATE]: Term 1 election: Requested vote from peers 0a7190f6535546cc9ebd48a3a4d6589b (127.5.71.189:36897), a1460ff13cfe410ea0458ae9774d957f (127.5.71.188:40785)
I20260923 20:52:26.846019 28502 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:26.846705 28502 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1460ff13cfe410ea0458ae9774d957f, State: Initialized, Role: FOLLOWER
I20260923 20:52:26.847417 28502 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:26.854787 28521 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:26.855368 28521 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:26.857685 28502 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:26.857750 28477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "587e49f036e94359b14c75527d0c720e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a1460ff13cfe410ea0458ae9774d957f"
I20260923 20:52:26.858465 28477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [term 0 FOLLOWER]: Advancing to term 1
W20260923 20:52:26.862877 28307 tablet.cc:2369] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 20:52:26.863569 28477 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 587e49f036e94359b14c75527d0c720e in term 1.
I20260923 20:52:26.865015 28302 leader_election.cc:302] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 587e49f036e94359b14c75527d0c720e, a1460ff13cfe410ea0458ae9774d957f; no voters:
I20260923 20:52:26.865729 28510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 20:52:26.866407 28510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [term 1 LEADER]: Becoming Leader. State: Replica: 587e49f036e94359b14c75527d0c720e, State: Running, Role: LEADER
I20260923 20:52:26.867129 28510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:26.873576 28531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587e49f036e94359b14c75527d0c720e. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:26.874150 28531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:26.877854 28535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 20:52:26.898487 28535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 20:52:26.900992 28542 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 20:52:26.906103 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.858066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.5.71.189:36897)
0923 20:52:26.858269 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.858276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.876691 (+ 18415us) server_negotiation.cc:407] Connection header received
0923 20:52:26.877318 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.877324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.877428 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.877528 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.881549 (+ 4021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.883076 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.885993 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.886952 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.887130 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.896632 (+ 9502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.896640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.896644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.896698 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.900701 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.902317 (+ 1616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.902323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.902326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.902505 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.903197 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.903229 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.903234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.903822 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.904210 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.905137 (+ 927us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.905518 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:52:26.918427 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.871266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.5.71.190:41735)
0923 20:52:26.871499 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.871508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.883197 (+ 11689us) server_negotiation.cc:407] Connection header received
0923 20:52:26.883476 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.883484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.883588 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.883692 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.888132 (+ 4440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.889557 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.902278 (+ 12721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.903021 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.903130 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.907847 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.907854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.907859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.907904 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.913863 (+ 5959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.915246 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.915252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.915257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.915435 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.916170 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.916179 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.916183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.916596 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.916881 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.917758 (+ 877us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.918023 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260923 20:52:26.923887 28427 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } attempt: 5
I20260923 20:52:26.906018 28286 master_runner.cc:186] Error getting master registration for 127.5.71.189:36897: OK, instance_id { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" instance_seqno: 1790196746191875 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 20:52:26.929189 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.860966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.5.71.188:40785)
0923 20:52:26.861236 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.861246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.881164 (+ 19918us) server_negotiation.cc:407] Connection header received
0923 20:52:26.886083 (+ 4919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.886092 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.886267 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.886406 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.889877 (+ 3471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.890988 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.893380 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.894135 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.894294 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.902152 (+ 7858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.902159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.902163 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.902220 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.920511 (+ 18291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.925473 (+ 4962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.925479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.925482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.925644 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.926116 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.926121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.926124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.926709 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.927242 (+ 533us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.928144 (+ 902us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.928570 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260923 20:52:26.929432 28286 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 20:52:26.934773 28477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [term 1 FOLLOWER]: Refusing update from remote peer 587e49f036e94359b14c75527d0c720e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:52:26.935829 28531 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 20:52:26.936777 28546 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 20:52:26.936908 28546 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 23 20:52:26 dist-test-slave-dn29 krb5kdc[28273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790196746, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.71.189@KRBTEST.COM for kudu/127.5.71.188@KRBTEST.COM
I20260923 20:52:26.954365 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.924470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.71.188:40785 (local address 127.0.0.1:57814)
0923 20:52:26.924731 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0923 20:52:26.924752 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 20:52:26.924879 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 20:52:26.925484 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 20:52:26.925491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 20:52:26.926159 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0923 20:52:26.926433 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.926450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.937267 (+ 10817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.937275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.938799 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 20:52:26.938818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.942543 (+ 3725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.942552 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 20:52:26.942813 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.943707 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 20:52:26.943734 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 20:52:26.946802 (+ 3068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 20:52:26.951271 (+ 4469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.951277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.951281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.951856 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.952362 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 20:52:26.952369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 20:52:26.952372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 20:52:26.952541 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 20:52:26.953304 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 20:52:26.953309 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 20:52:26.953475 (+ 166us) client_negotiation.cc:769] Sending connection context
0923 20:52:26.953641 (+ 166us) client_negotiation.cc:240] Negotiation successful
0923 20:52:26.953937 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260923 20:52:26.955379 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:26.924514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.5.71.188:40785)
0923 20:52:26.924706 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:26.924714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:26.924812 (+ 98us) server_negotiation.cc:407] Connection header received
0923 20:52:26.925019 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:26.925028 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:26.925130 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:26.925239 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:26.927212 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.936584 (+ 9372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.938953 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:26.939636 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:26.939825 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:26.947020 (+ 7195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 20:52:26.947027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 20:52:26.947032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 20:52:26.947071 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0923 20:52:26.951006 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.952013 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.952019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.952023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.952204 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 20:52:26.952689 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 20:52:26.952694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 20:52:26.952697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0923 20:52:26.953074 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 20:52:26.953238 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:26.954742 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0923 20:52:26.954982 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 20:52:26.961257 28427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: Bootstrap starting.
I20260923 20:52:26.934864 28435 master_runner.cc:186] Error getting master registration for 127.5.71.189:36897: OK, instance_id { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" instance_seqno: 1790196746191875 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 20:52:26.965629 28435 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 20:52:26.965796 28427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: Neither blocks nor log segments found. Creating new log.
I20260923 20:52:26.966454 28521 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587e49f036e94359b14c75527d0c720e. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:26.966747 28521 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:26.970953 28427 log.cc:824] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: Log is configured to *not* fsync() on all Append() calls
I20260923 20:52:26.984565 28531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:26.984967 28531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:26.987732 28535 catalog_manager.cc:1347] Generated new cluster ID: 306ac0a027264da98ad79f564ee0cc6b
I20260923 20:52:26.987881 28535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 20:52:26.989257 28521 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:26.989508 28521 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:26.991863 28427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: No bootstrap required, opened a new log
I20260923 20:52:27.005905 28427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:27.009178 28427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7190f6535546cc9ebd48a3a4d6589b, State: Initialized, Role: FOLLOWER
I20260923 20:52:27.009941 28427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } }
I20260923 20:52:27.011524 28553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:27.012416 28553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:27.012912 28427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 20:52:27.012962 28402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "587e49f036e94359b14c75527d0c720e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0a7190f6535546cc9ebd48a3a4d6589b"
I20260923 20:52:27.013460 28402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [term 0 FOLLOWER]: Advancing to term 1
I20260923 20:52:27.017091 28402 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 587e49f036e94359b14c75527d0c720e in term 1.
I20260923 20:52:27.018106 28401 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [term 1 FOLLOWER]: Refusing update from remote peer 587e49f036e94359b14c75527d0c720e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 20:52:27.019105 28531 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 20:52:27.055262 28535 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 20:52:27.057431 28535 catalog_manager.cc:1504] Loading token signing keys...
I20260923 20:52:27.062700 28553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587e49f036e94359b14c75527d0c720e. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:27.063071 28553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:27.063657 28560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:27.064174 28560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 20:52:27.065498 28510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "587e49f036e94359b14c75527d0c720e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587e49f036e94359b14c75527d0c720e" member_type: VOTER last_known_addr { host: "127.5.71.190" port: 41735 } } peers { permanent_uuid: "0a7190f6535546cc9ebd48a3a4d6589b" member_type: VOTER last_known_addr { host: "127.5.71.189" port: 36897 } } peers { permanent_uuid: "a1460ff13cfe410ea0458ae9774d957f" member_type: VOTER last_known_addr { host: "127.5.71.188" port: 40785 } } }
I20260923 20:52:27.065869 28510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e [sys.catalog]: This master's current role is: LEADER
I20260923 20:52:27.094473 28535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 587e49f036e94359b14c75527d0c720e: Generated new TSK 0
I20260923 20:52:27.095963 28535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 20:52:27.122922 28568 catalog_manager.cc:1259] Loaded cluster ID: 306ac0a027264da98ad79f564ee0cc6b
I20260923 20:52:27.123047 28568 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: loading cluster ID for follower catalog manager: success
I20260923 20:52:27.126951 28568 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: acquiring CA information for follower catalog manager: success
I20260923 20:52:27.135354 28568 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a7190f6535546cc9ebd48a3a4d6589b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260923 20:52:27.260126 28432 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.237288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.5.71.189:36897)
0923 20:52:27.237534 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.237541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.256658 (+ 19117us) server_negotiation.cc:407] Connection header received
0923 20:52:27.258189 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.258197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.258327 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.258416 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.259834 (+ 1418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43786: BlockingRecv error: recv got EOF from 127.0.0.1:43786 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260923 20:52:27.262359 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.238413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.5.71.188:40785)
0923 20:52:27.238664 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.238672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.257707 (+ 19035us) server_negotiation.cc:407] Connection header received
0923 20:52:27.260534 (+ 2827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.260542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.260666 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.260743 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.262021 (+ 1278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57830: BlockingRecv error: recv got EOF from 127.0.0.1:57830 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260923 20:52:27.273416 28506 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.248636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.5.71.190:41735)
0923 20:52:27.248895 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.248903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.263469 (+ 14566us) server_negotiation.cc:407] Connection header received
0923 20:52:27.263641 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.263647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.263751 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.263902 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.273148 (+ 9246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37992: BlockingRecv error: recv got EOF from 127.0.0.1:37992 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260923 20:52:27.328269 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.309659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.5.71.189:36897)
0923 20:52:27.310018 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.310024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.320088 (+ 10064us) server_negotiation.cc:407] Connection header received
0923 20:52:27.320483 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.320491 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.320725 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.320881 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.321653 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.322857 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.325001 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.325712 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.325816 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.326616 (+ 800us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.327379 (+ 763us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.327519 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.327807 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.327914 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
W20260923 20:52:27.368893 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.355705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.5.71.188:40785)
0923 20:52:27.355947 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.355954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.366614 (+ 10660us) server_negotiation.cc:407] Connection header received
0923 20:52:27.367200 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.367211 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.367358 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.367447 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.368623 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57834: BlockingRecv error: recv got EOF from 127.0.0.1:57834 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260923 20:52:27.423926 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.405454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.5.71.189:36897)
0923 20:52:27.405641 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.405647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.416501 (+ 10854us) server_negotiation.cc:407] Connection header received
0923 20:52:27.417152 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.417159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.417299 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.417383 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.418098 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.419297 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.421445 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.422187 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.422289 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.422974 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.423141 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.423206 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.423435 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.423526 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260923 20:52:27.471872 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.458788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57846 (local address 127.5.71.188:40785)
0923 20:52:27.458988 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.458995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.469704 (+ 10709us) server_negotiation.cc:407] Connection header received
0923 20:52:27.470249 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.470255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.470360 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.470448 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.471590 (+ 1142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57846: BlockingRecv error: recv got EOF from 127.0.0.1:57846 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 20:52:27.535017 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.516695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.5.71.189:36897)
0923 20:52:27.516890 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.516896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.527742 (+ 10846us) server_negotiation.cc:407] Connection header received
0923 20:52:27.528326 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.528333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.528450 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.528523 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.529289 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.530434 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.532520 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.533267 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.533364 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.534068 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.534231 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.534307 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.534549 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.534657 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260923 20:52:27.582963 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.570044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57858 (local address 127.5.71.188:40785)
0923 20:52:27.570223 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.570229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.580805 (+ 10576us) server_negotiation.cc:407] Connection header received
0923 20:52:27.581392 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.581398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.581510 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.581593 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.582714 (+ 1121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57858: BlockingRecv error: recv got EOF from 127.0.0.1:57858 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 20:52:27.641501 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.622892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.5.71.189:36897)
0923 20:52:27.623091 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.623097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.634176 (+ 11079us) server_negotiation.cc:407] Connection header received
0923 20:52:27.634536 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.634543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.634650 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.634734 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.635444 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.636629 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.638954 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.639814 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.639930 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.640583 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.640755 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.640824 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.641095 (+ 271us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.641188 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260923 20:52:27.668185 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.665953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.5.71.188:40785)
0923 20:52:27.666247 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.666254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.666536 (+ 282us) server_negotiation.cc:407] Connection header received
0923 20:52:27.666751 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.666756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.666839 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.666938 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.667975 (+ 1037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57868: BlockingRecv error: recv got EOF from 127.0.0.1:57868 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
I20260923 20:52:27.686748 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.679853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.5.71.189:36897)
0923 20:52:27.680080 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.680087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.680546 (+ 459us) server_negotiation.cc:407] Connection header received
0923 20:52:27.680752 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.680758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.680857 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.680974 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.681455 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.682639 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.684434 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.685150 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.685296 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.685804 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.685965 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.686070 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.686294 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.686390 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260923 20:52:27.696023 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.693732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.5.71.188:40785)
0923 20:52:27.693987 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.693995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.694400 (+ 405us) server_negotiation.cc:407] Connection header received
0923 20:52:27.694734 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.694739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.694815 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.694925 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.695751 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57882: BlockingRecv error: recv got EOF from 127.0.0.1:57882 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
I20260923 20:52:27.723104 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.716709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.5.71.189:36897)
0923 20:52:27.716932 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.716940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.717513 (+ 573us) server_negotiation.cc:407] Connection header received
0923 20:52:27.717782 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.717786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.717875 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.717968 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.718435 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.719260 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.721285 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.721786 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.721902 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.722379 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.722486 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.722556 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.722713 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.722782 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260923 20:52:27.731681 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.729578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.5.71.188:40785)
0923 20:52:27.729791 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.729797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.730151 (+ 354us) server_negotiation.cc:407] Connection header received
0923 20:52:27.730303 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.730309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.730423 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.730544 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.731438 (+ 894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57890: BlockingRecv error: recv got EOF from 127.0.0.1:57890 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 20:52:27.772588 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.767300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.5.71.189:36897)
0923 20:52:27.767549 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.767557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.767913 (+ 356us) server_negotiation.cc:407] Connection header received
0923 20:52:27.768145 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.768150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.768233 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.768341 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.768735 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.769464 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.770766 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.771243 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.771398 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.771834 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.771959 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.772057 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.772202 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.772283 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260923 20:52:27.780573 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.778389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57894 (local address 127.5.71.188:40785)
0923 20:52:27.778558 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.778563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.779026 (+ 463us) server_negotiation.cc:407] Connection header received
0923 20:52:27.779199 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.779203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.779286 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.779375 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.780334 (+ 959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57894: BlockingRecv error: recv got EOF from 127.0.0.1:57894 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 20:52:27.853878 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.848545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.5.71.189:36897)
0923 20:52:27.848769 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.848774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.849085 (+ 311us) server_negotiation.cc:407] Connection header received
0923 20:52:27.849248 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.849252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.849325 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.849419 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.849896 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.850657 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.852053 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.852569 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.852712 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.853126 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.853263 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.853344 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.853577 (+ 233us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.853655 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260923 20:52:27.862429 28505 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 20:52:27.860177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57898 (local address 127.5.71.188:40785)
0923 20:52:27.860403 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.860411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.860722 (+ 311us) server_negotiation.cc:407] Connection header received
0923 20:52:27.860901 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.860904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.860992 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.861095 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0923 20:52:27.862169 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57898: BlockingRecv error: recv got EOF from 127.0.0.1:57898 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 20:52:27.939074 28546 catalog_manager.cc:1259] Loaded cluster ID: 306ac0a027264da98ad79f564ee0cc6b
I20260923 20:52:27.939178 28546 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: loading cluster ID for follower catalog manager: success
I20260923 20:52:27.942276 28546 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: acquiring CA information for follower catalog manager: success
I20260923 20:52:27.944222 28546 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a1460ff13cfe410ea0458ae9774d957f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 20:52:27.999920 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:27.994346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.5.71.189:36897)
0923 20:52:27.994718 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:27.994726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:27.995104 (+ 378us) server_negotiation.cc:407] Connection header received
0923 20:52:27.995198 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:27.995203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:27.995282 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:27.995378 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:27.996025 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.996795 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.998118 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:27.998589 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:27.998696 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:27.999106 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:27.999226 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:27.999300 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:27.999500 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 20:52:27.999568 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260923 20:52:28.012841 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:28.006083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.5.71.188:40785)
0923 20:52:28.006382 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:28.006390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:28.006774 (+ 384us) server_negotiation.cc:407] Connection header received
0923 20:52:28.007026 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:28.007031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:28.007222 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:28.007367 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:28.007846 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.008744 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.010244 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.010721 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.010879 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:28.011501 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:28.012155 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:28.012352 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:28.012409 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 20:52:28.012509 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260923 20:52:28.029872 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:28.019614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38006 (local address 127.5.71.190:41735)
0923 20:52:28.022185 (+ 2571us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:28.022193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:28.022217 (+ 24us) server_negotiation.cc:407] Connection header received
0923 20:52:28.022357 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:28.022364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:28.022626 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:28.022815 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:28.023828 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.025140 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.027238 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.027674 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.027844 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:28.028451 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:28.028966 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:28.029068 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:28.029306 (+ 238us) server_negotiation.cc:299] Negotiation successful
0923 20:52:28.029398 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2416,"thread_start_us":2290,"threads_started":1}
I20260923 20:52:28.030865 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:28.020059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43886 (local address 127.5.71.189:36897)
0923 20:52:28.020302 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:28.020311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:28.022580 (+ 2269us) server_negotiation.cc:407] Connection header received
0923 20:52:28.022822 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:28.022827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:28.022917 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:28.022985 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:28.024019 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.025173 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.028286 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.028857 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.028943 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:28.029911 (+ 968us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:28.030063 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:28.030160 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:28.030350 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 20:52:28.030462 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260923 20:52:28.030992 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 20:52:28.019566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57920 (local address 127.5.71.188:40785)
0923 20:52:28.019909 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0923 20:52:28.019917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 20:52:28.023648 (+ 3731us) server_negotiation.cc:407] Connection header received
0923 20:52:28.023951 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 20:52:28.023957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 20:52:28.024063 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 20:52:28.024170 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 20:52:28.024662 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.025514 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.028257 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 20:52:28.028908 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 20:52:28.029058 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 20:52:28.030245 (+ 1187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 20:52:28.030367 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 20:52:28.030467 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 20:52:28.030621 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 20:52:28.030701 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260923 20:52:28.037528 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 28286
I20260923 20:52:28.038856 28348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:28.155575 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 28286
W20260923 20:52:28.177129 28378 connection.cc:535] server connection from 127.0.0.1:43734 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 20:52:28.177505 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 28360
I20260923 20:52:28.178694 28422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:28.290783 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 28360
I20260923 20:52:28.312093 5406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQIJ804/build/asan/bin/kudu pid 28435
I20260923 20:52:28.313480 28497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260923 20:52:28.424530 5406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQIJ804/build/asan/bin/kudu with pid 28435
2026-09-23T20:52:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4693 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9876 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (388198 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST